Shared Memory

Shared Memory is memory that can be seen by more than one process. Normally, when you fork a process, the 2 processes don't access the same memory. Shared Memory is another IPC resource.

You acquire a shared memory segment by making a call that looks like:
shmid = shmget(key, size/* in bytes */, flags);

Many shared memory concepts are the same as semaphores. For example:

  1. the key must be unique on a given host
  2. the key can be IPC_PRIVATE if you want shared memory that isn't associated with any key.
  3. The flags include permissions and can optionally contain IPC_CREAT and IPC_EXCL. IPC_CREAT creates a shared memory segment if it doesn't already exist. If IPC_EXCL is included with IPC_CREAT then it is considered a failure if the shared memory does already exist. Some valid flag combinations are:
    0660
    0660 | IPC_CREAT
    0600 | IPC_CREAT | IPC_EXCL
  4. A negative return code indicates failure and you should check errno (using perror or strerror).
  5. Normally, your program should free up any shared memory that it allocates. However, if your program crashes, your shared memory doesn't automatically get freed up. It stays around and can be seen with the ipcs command.
  6. You can free up your shared memory with a command like:
    ipcrm shm xxxxx
  7. The script below will destroy all of your shared memory segments:

remove_shm

#!/bin/csh -f
set l = `ipcs -m | grep "$USER"| cut -c12-19`
foreach s ( $l )
    echo $s
    ipcrm shm $s 
end
if ($#l != 0 ) echo $#l shared memory\(s\) for $user removed

shmat

Once you have a shared memory segment id (shmid), then you next need to to map the memory into your address space. Normally, this should be done with the command

char * shm_addr;

shm_addr = shmat( shmid, NULL, 0);

if (shm_addr == (char *) -1) you have an error.

If you want READ_ONLY access to your shared memory, you can set the third parameter to SHM_RDONLY.

It is also possible to set the second parameter to the address where you want your shared memory to show up. I would recommend that you don't use this option. The reason people might want this is because they have pointers in their shared memory. This is a bad choice. Normally, pointers can be avoided by properly designing your data structures.

When you get back a good shm_addr, remember it and normally you will want to caste it to data type of your shared memory. Usually its a good idea to create a struct that contains your shared memory. In this way, you can get the compiler to do most of the work computing sizes and figuring out pointers, etc.

Once you have gotten this far, you are ready to go.

The normal reason for shared memory is because you have 2 processes that want to share information. This also means that you need to very carefully think through the synchronization issues. Often shared memory and semaphores go together.

 

When you are all done, you need to free up your shared memory. Remember that if you don't do this before you exit, you will see the resource in ipcs.

Freeing up shared memory:

First you should detach the memory from your address space (inverse of shmat) with a command like:
shmdt(shm_addr);

To get rid of the ipcs shared memory resource, issue the command:

shmctl(shmid, IPC_RMID, 0);

 

shm_com.h

/* A common header file to describe the shared memory we wish to pass about. */

#define TEXT_SZ 2048

struct shared_use_st {
    int written_by_you;
    char some_text[TEXT_SZ];
};

shm1.c

/* Our first program is a consumer. After the headers the shared memory segment
 (the size of our shared memory structure) is created with a call to shmget,
 with the IPC_CREAT bit specified. */

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>

#include "shm_com.h"

int main()
{
    int running = 1;
    void *shared_memory = (void *)0;
    struct shared_use_st *shared_stuff;
    int shmid;
    int mykey = getuid();

    srand((unsigned int)getpid());    

    shmid = shmget((key_t)mykey, sizeof(struct shared_use_st), 0666 | IPC_CREAT);

    if (shmid == -1) {
        fprintf(stderr, "shmget failed\n");
        exit(EXIT_FAILURE);
    }

/* We now make the shared memory accessible to the program. */

    shared_memory = shmat(shmid, (void *)0, 0);
    if (shared_memory == (void *)-1) {
        fprintf(stderr, "shmat failed\n");
        exit(EXIT_FAILURE);
    }

    printf("Memory attached at %X\n", (int)shared_memory);

/* The next portion of the program assigns the shared_memory segment to shared_stuff,
 which then prints out any text in written_by_you. The loop continues until end is found
 in written_by_you. The call to sleep forces the consumer to sit in its critical section,
 which makes the producer wait. */

    shared_stuff = (struct shared_use_st *)shared_memory;
    shared_stuff->written_by_you = 0;
    while(running) {
        if (shared_stuff->written_by_you) {
            printf("You wrote: %s", shared_stuff->some_text);
            sleep( rand() % 4 ); /* make the other process wait for us ! */
            shared_stuff->written_by_you = 0;
            if (strncmp(shared_stuff->some_text, "end", 3) == 0) {
                running = 0;
            }
        }
    }

/* Lastly, the shared memory is detached and then deleted. */

    if (shmdt(shared_memory) == -1) {
        fprintf(stderr, "shmdt failed\n");
        exit(EXIT_FAILURE);
    }

    if (shmctl(shmid, IPC_RMID, 0) == -1) {
        fprintf(stderr, "shmctl(IPC_RMID) failed\n");
        exit(EXIT_FAILURE);
    }

    exit(EXIT_SUCCESS);
}


shm2.c

/* The second program is the producer and allows us to enter data for consumers.
 It's very similar to shm1.c and looks like this. */

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>

#include "shm_com.h"

int main()
{
    int running = 1;
    void *shared_memory = (void *)0;
    struct shared_use_st *shared_stuff;
    char buffer[TEXT_SZ ];
    int shmid;
    int mykey = getuid();


    shmid = shmget((key_t)mykey, sizeof(struct shared_use_st), 0666 | IPC_CREAT);

    if (shmid == -1) {
        fprintf(stderr, "shmget failed\n");
        exit(EXIT_FAILURE);
    }

    shared_memory = shmat(shmid, (void *)0, 0);
    if (shared_memory == (void *)-1) {
        fprintf(stderr, "shmat failed\n");
        exit(EXIT_FAILURE);
    }

    printf("Memory attached at %X\n", (int)shared_memory);

    shared_stuff = (struct shared_use_st *)shared_memory;
    while(running) {
        while(shared_stuff->written_by_you == 1) {
            sleep(1);            
            printf("waiting for client...\n");
        }
        printf("Enter some text: ");
        fgets(buffer, BUFSIZ, stdin);
        
        strncpy(shared_stuff->some_text, buffer, TEXT_SZ);
        shared_stuff->written_by_you = 1;

        if (strncmp(buffer, "end", 3) == 0) {
                running = 0;
        }
    }

    if (shmdt(shared_memory) == -1) {
        fprintf(stderr, "shmdt failed\n");
        exit(EXIT_FAILURE);
    }
    exit(EXIT_SUCCESS);
}

Experiments to try out

What happens if you call shmctl to remove the shared memory at the end of shm2.c?

What happens if shm1.c calls shmctl to remove the shared memory and shm2 continues to use the shared memory. Add some code in shm2.c to delay before detaching (shmdt). Do an ipcs to see if the shared memory is gone.

What happens if shm2.c forgets to detach the shared memory (shmdt)


shcopy.c

This program attempts to provide a way to copy an input into an output in such a way as to have the reads and writes done in parallel. The buffers live in shared memory and the synchronization between the 2 tasks occurs with semaphores.

This program creates data structures in shared memory to support multiple buffers. It uses 2 semaphores to keep a count of the number of full buffers and the number of empty buffers.

A Reader task is started to read as fast as possible into free buffers. The Reader needs to do the following:

A Writer task is started to write as fast as possible from the full buffers. The Writer needs to do the following:

The main routine sets up the shared memory and the semaphore. Then the main routine creates the Reader and Writer tasks. The main routine waits for the completion of the Reader and the Writer before freeing up resources.

I ran into a problem when using the SEM_UNDO flag. It seems that if the reader task terminates before the writer and all of the semaphore operations use SEM_UNDO, the semaphore counters get messed up as the reader terminates. Removing this flag in favor of a zero fixes the problem. If you really want the SEM_UNDO flag set, then I needed a way to keep the reader task around until the writer finished. Note the commented out code at the end of the reader that waits for the buffers to be freed up again.

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/sem.h>
#include <sys/stat.h>
#include <fcntl.h>

// ****************************************************
// GLOBAL definitions and data
// **************************************************** 


// The following flag turns on debug compilation code
#define DEBUG 1

union semun {
        int val;                    /* value for SETVAL */
        struct semid_ds *buf;       /* buffer for IPC_STAT, IPC_SET */
        unsigned short int *array;  /* array for GETALL, SETALL */
        struct seminfo *__buf;      /* buffer for IPC_INFO */
    };

// In real life, you would probably make both of the following numbers
// much larger
#define BUFF_SIZE 1024
#define NUM_BUFFS 2

// Shared Memory data structure

struct big_buff
{
    int numBytes;
    char buff[BUFF_SIZE];
};
typedef struct big_buff BIG_BUFF;

struct copy_buffs
{
    int get;
    int put;
    int numRead;
    int numWritten;
    int done;
    BIG_BUFF  buffers[NUM_BUFFS];
};
int shmid = -1;
void *shared_memory = (void *)0;

int semid = -1; /* Semaphore IPC ID */


// ****************************************************
// Setup the Semaphores
// **************************************************** 

int setup_semid()
{ 
    union semun semarg;
    int mykey = getuid();
    unsigned short  vals[] = {NUM_BUFFS/*free buffers*/,   0/*full buffers*/};
    semid = semget(mykey, 2, 0666 | IPC_CREAT  | IPC_EXCL);
    if (semid < 0)
    {
        perror("semget(semid)");
        return -1;
    } 
    printf("created semid %d\n", semid);

    semarg.array = vals;
    if (semctl(semid, 0, SETALL, semarg) < 0)
    {
        perror("semctl(semid)");
        free_resources();
        return -1;
    }
    return 0;
}

// ****************************************************
// Used to increment and decrement Free Buffer counters
// **************************************************** 

void accessFreeBuffs(int freeBuff_inc){

       struct sembuf sops;
#ifdef DEBUG
       // This debug code was added to watch the semaphore counters in action

       union semun semarg;
       unsigned short vals[2];
       
       semarg.array = vals;
       if (semctl(semid, 0, GETALL, semarg) < 0)
        {
             perror("accessFreeBuffs: semctl");
        }
        
       printf("accessFreeBuffs freeBuff_inc(%d) val[free] = %d  val[full]=%d\n", 
            freeBuff_inc, vals[0], vals[1]);
#endif
       sops.sem_flg = 0; //SEM_UNDO -- causes problems 
                         // when Reader terminates before writer;
    
       sops.sem_num = 0;
       sops.sem_op = freeBuff_inc;
       if (semop(semid, &sops, 1) < 0)
                perror("semop buffs");    
       return;
   }
   
// ****************************************************
// Used to decrement and increment Full Buffer counters
// **************************************************** 

void accessFullBuffs(int fullBuff_inc){
       struct sembuf sops;
#ifdef DEBUG
 // This debug code was added to watch the semaphore counters in action

       union semun semarg;
       unsigned short vals[2];
       
       semarg.array = vals;
       if (semctl(semid, 0, GETALL, semarg) < 0)
        {
             perror("accessFreeBuffs: semctl");
        }
        
       printf("accessFullBuffs fullBuff_inc(%d) val[free] = %d  val[full]=%d\n", 
            fullBuff_inc, vals[0], vals[1]);
#endif
       sops.sem_flg = 0; //SEM_UNDO ; Causes problems 
                         // when reader terminates before writer
    
       sops.sem_num = 1;
       sops.sem_op = fullBuff_inc;
       if (semop(semid, &sops, 1) < 0)
                perror("semop buffs");    
       return;
   }


// **************************************************
// Setup and initialize Shared Memory
// **************************************************** 

struct copy_buffs * setup_shared_mem()
{    
    int mykey = getuid();
    printf("My shared memory key is %d\n", mykey);

    shmid = shmget((key_t)mykey, sizeof(struct copy_buffs), 0666 | 
                    IPC_CREAT | IPC_EXCL);

    if (shmid == -1) {
        perror("shmget failed\n");
        exit(-1);
    }
    
    shared_memory = shmat(shmid, (void *)0, 0);
    if (shared_memory == (void *)-1) {
        perror("shmat failed\n");
        exit(-1);
    }
// Good idea to initialize shared memory
    memset(shared_memory, 0, sizeof(struct copy_buffs));

    return (struct copy_buffs *) shared_memory;
}
// ****************************************************
// Free up semaphore and shared memory resources. 
// **************************************************** 

free_resources()
{
    union semun semarg;
// Detach Memory address
    if (shmdt(shared_memory) == -1) {
        perror("shmdt failed\n");
    }

    if (shmctl(shmid, IPC_RMID, 0) == -1) {
        perror("shmctl(IPC_RMID) failed\n");
    }
 
    if (semctl(semid, 0, IPC_RMID, semarg) < 0)
    {
        perror("free_resources: semid ");
    }
}
// ****************************************************
// Reader task decrements the free buffer semaphore and reads into it.  
// Once the buffer is read, the full buffer semaphore is incremented.
// **************************************************** 

void reader(struct copy_buffs *shared_stuff, char * filename)
{
    int fid;
    int nread, index;
    BIG_BUFF * buff_ptr;  
    char *buff;  
    int count=0;

    fid = open(filename, O_RDONLY);
    if (fid < 0)
        {
            printf("reader open err(%s): %s\n", filename, strerror(errno));
            printf("reader terminating\n");
            shared_stuff->done = 1;            
            accessFullBuffs(1); //  add 1 "full"  buffer

            return;
        }

    do
    {
        accessFreeBuffs(-1); // Need one free buffer 

        index = shared_stuff->get;
        buff_ptr = &shared_stuff->buffers[index];
        buff = buff_ptr->buff;
        shared_stuff->get = (shared_stuff->get +1) % NUM_BUFFS;
        nread = read(fid, buff_ptr, BUFF_SIZE);
        buff_ptr->numBytes = nread;
        if (nread > 0)
            {
#ifdef DEBUG
                printf("reader read another buffer %d\n", count++);
#endif
                shared_stuff->numRead += 1;
                accessFullBuffs(1); // add 1 full buffers
            }
        else if (nread <= 0)
            {    
                printf("reader terminated with a read of 0 bytes\n");
                shared_stuff->done = 1;            
                accessFullBuffs(1); //  add 1 full buffers
          
                if (nread < 0) perror("read error with nread < 0)" );
            }
            
    } while (nread > 0);

    close(fid);
   
// If I uncomment out the lines that use the SEM_UNDO option, then
// I need the following lines uncommented.  It seems like with SEM_UNDO
// set, that when the reader task terminates, the counters in the semaphore
// get messed up.  So the following lines can be used to keep the Reader from
// terminating until the Writer has freed up all of his buffers.
      
//    accessFreeBuffs(-1); 
//    accessFreeBuffs(-1); 
}

// ****************************************************
// The writer decrements the Full buffer semaphore and then writes it
// to the appropriate file.  Then the writer increments the full buffer semaphore.
// **************************************************** 

void writer(struct copy_buffs *shared_stuff, char * filename)
{
    int index, fid, nwritten;
    BIG_BUFF * buff_ptr;
    char *buff;    
    int count = 0;

    unlink(filename);
    fid = open(filename, O_WRONLY|O_CREAT| S_IRUSR |S_IWUSR );
    if (fid < 0)
        {
            printf("writer open err(%s): %s\n", filename, strerror(errno));
            return;
        }
    
    do
    {
        accessFullBuffs( -1); // Need one full buffers

        if (shared_stuff->done)
        {
            printf("writer sees done flag read(%d) written(%d)\n",
                shared_stuff->numRead, shared_stuff->numWritten);
            
            if (shared_stuff->numRead == shared_stuff->numWritten)
            {
                accessFreeBuffs(1); // free buffer 
                break;
            }
        }
        index = shared_stuff->put;
        buff_ptr = &shared_stuff->buffers[index];
        buff = buff_ptr->buff;
        shared_stuff->put = (shared_stuff->put +1) % NUM_BUFFS;
        nwritten = write(fid, buff, buff_ptr->numBytes);
#ifdef DEBUG
        printf("writer wrote another block %d\n", count++);
#endif
        if (nwritten > 0)
            {
                shared_stuff->numWritten += 1;
                accessFreeBuffs(1); // free one buffer 
            }
        else if (nwritten <= 0)
            {    
                perror("write error with nwritten < 0)" );
            }
            
    } while (nwritten > 0);

    close(fid);
}


// **************************************************** 
// Initializes shared memory, and the Semaphores.
// Then main starts the reader and writer tasks.
// Main waits for the termination of the reader and the 
// writer before freeing up all of the resources.  
// **************************************************** 

int main(int argc, char * argv[])
{
    struct copy_buffs *shared_stuff;
    int i, pid_reader, pid_writer, pid, status;

    if (argc < 3)
     {
        printf("usage: %s infile outfile\n");
        exit(0);
     }
        
    shared_stuff = setup_shared_mem();
    if (setup_semid())
     {
        free_resources();
     }

    pid_reader = fork();
    if (pid_reader == 0)
     {
        reader(shared_stuff, argv[1]);
        exit(0);
     }
    if (pid_reader < 0)
     {
        free_resources();
        perror("reader fork error");
        exit(-1);
     }
    printf("Created reader task %d\n", pid_reader);


    pid_writer = fork();
    if (pid_writer == 0)
     {
        writer(shared_stuff, argv[2]);
        exit(0);
     }
    if (pid_writer < 0)
     {
        free_resources();
        perror("reader fork error");
        exit(-1);
     }

    printf("Created writer task %d\n", pid_writer);


    for (i=0; i < 2; i++)
        {
            pid = wait(&status);
            if (pid == pid_reader)
                printf("reader terminated\n");
            else if (pid == pid_writer)
                printf("writer terminated\n");
            else
                printf(" Some pid terminated %d\n");            
        }
    printf("Everything done\n");
    

    free_resources();

}    


Battleship program

This program has a little too much complexity to go over, but it does use semaphores and shared memory. I want to demo it, because it gives me an opportunity to talk about archiving a project.

battleship.tar.gz

Download this file and ftp it up to some directory on Unix. Issue the command:
tar zxvf battleship.tar.gz

make

To start a new game, issue the command:

./bttleship

To join a game find out your opponents game number (pretend like it's 284913) and issue the command:

./bttleship 284913

You enter 2 letters and hit enter to throw a bomb. Note that you can bomb your own boats if you aren't careful.

We can demo this in class.


Exercise

Go back to the dining philosopher program and modify it to have some shared memory that keeps track of which philosopher is eating. Then modify the display routine to put out smiley faces on all of the philosophers that are eating and blanks when they are thinkinig.