Accessing Arduino Program Memory

I suspect that for most users of Arduinos, the limited amount of data RAM (ATMega328 has 2KB) is a cause of concern, and with more complex sketches or for those using many libraries, a real problem. Now while it is often the case that there is plenty of program memory available (ATMega328 has 32KB) , the architecture of the ATM chip-set does not allow immediate access to this space. Yet there are many times when you need to store many constants and / or arrays of data or text (such as LCD character sets, MIDI program information, menus etc.) and it would be great to have these tucked away in program memory and not using any precious RAM.

Enter PROGMEM.

While there are libraries of functions (Mikal Hart’s FLASH library, for one) to simplify the use of storing and reading data in the program memory, I like to understand what’s happening under the hood, so I hope you find this discussion helpful.

PROGMEM is part of the pgmspace.h library that comes with the Arduino IDE software. PROGMEM tells the compiler that data (of a given datatype) is to be stored in program memory. While there are many discussions and examples to help users use the PROGMEM macros and functions, I have often found them to be confusing and to not always cover specifically what I wanted to do. I’ve shamelessly plagerized from multiple online articles and created the following discussion. Hopefully this article will help to clarify this useful function.

Below is a small sketch that exercises the PROGMEM declarations and special functions and macros to store and retrieve examples of several different data types. These include a 2D arrays of bytes used to make custom LCD characters (see commented out code), a 1D array of integers, an array of character strings (for a menu, for example), a long character string and an array of floating point numbers (for fixed GPS coordinates, for example). These are stored and accessed in turn and displayed on the serial terminal.

Storing bytes, integers and floats: This simply requires adding PROGMEM to the end of the datatype declaration, as follows. Note that “const” tells the compiler that the data is read-only.

// 2D byte array (for LCD custom characters, for instance)
const char charBitmap [][8] PROGMEM = {              
    { 0x00,0x00,0x00,0x00,0x00,0x07,0x07,0x07 },
    { 0x00,0x00,0x00,0x00,0x00,0x1C,0x1C,0x1C },
     ....
    { 0x00,0x00,0x07,0x07,0x07,0x00,0x00,0x00 }
};

// Array of unsigned integers stored in PROGMEM
const unsigned int uintSet[] PROGMEM = { 65535, 32796, 16843, 10, 11234};

// Array of floats
const float fSet[] PROGMEM = {1.19278, 0.017452, 3.14159};

// Long piece of text
const char LongMes1[] PROGMEM  = {"The quick brown fox jumped over the lazy brown dog"};
const char LongMes2[] PROGMEM  = {"Never in the history of man has so much been owed by so many to so few"};

 

The special case of storing arrays of strings:  When storing an array of strings (which themselves are arrays of characters), there is an additional level of indirection to consider. Each string is stored separately in program memory. In addition, a string table is also created. This is an array of pointers (hence the use of the pointer *) to the strings, as follows:

(UPDATE: To pass Arduino IDE 1.6 verification, the correct syntax to refer to an array of pointer strings to be held in program memory is as follows: )

const char * const NAME_OF_ARRAY [] PROGMEM = { String1, String2, etc. };
const char Line1  [] PROGMEM = "\tPROGMEM MEMORY TESTS";
const char Line2  [] PROGMEM = "\tFree Memory:  ";
const char Line3  [] PROGMEM = "B ";

const char * const Lines [] PROGMEM = { Line1, Line2, Line2 };

To access the nth string, you first access the nth pointer in the string table. This pointer contains the program memory address of the start of the nth string. You then copy characters from this address until the end of the string (which is always terminated with 0x0) into a buffer using strcpy_P function (a version of the strcpy function modified to access program memory space) .

char buffer[x];      // make as large as the longest string
strcpy_P(buffer, (char*) pgm_read_word( &Lines[n] ) );

The string is now held in the buffer and you can print or manipulate further.

Reading bytes, words and floats:  The sketch also uses the macros “pgm_read_byte(&x)”, “pgm_read_word(&x)” and “pgm_read_float(&x)” to read bytes, integers and floats respectively from program memory. These special macros require a parameter containing the address of the pointer to the element in program memory – hence the use of the “&” dereference – to access its contents. One way to think about using the pgm_read_* macros is to start from the situation if you were accessing a location in SRAM and follow these steps:

1. Normal access in SRAM:   someByte = someData[x][y];
2. Take the data address:   someByte = &( someData[x][y] ); 
3. Add pgm_read_* function: someByte = pgm_read_byte ( &( someData[x][y] ) );

The “pgm_read_byte_near(element + x)”, and companion set of pgm_read_word_near() and pgm_read_float_near() macros, take a parameter containing the pointer to the initial element of the array and an offset (automatically sized to the type definition).

The pgm_read_* and pgm_read_*_near macros make it simple to access individual elements of an array, no matter whether it is an array of bytes/characters, strings, integers, longs or floats. See here for a full description of the type definitions, macros and functions available in the Program Space Utilities pgmspace.h library.

Note that you will find that this sketch only uses only 17 bytes of SRAM: 16 for the character buffer and 1 for a byte variable. (However a further 16 bytes appears to be used by the pgm_read_float() function…  I don’t know why yet.) Compared to having these arrays in SRAM this is a HUGE saving for very little additional coding.

Anyway, here’s hoping that this helps to make it a little clearer as to how to use the PROGMEM function and save lots of precious RAM in the future.

Here’s the ARDUINO code:

[codesyntax lang=”php” title=”Accessing Arduino Program Memory” blockstate=”collapsed”]

/*******************************************************************************************
* /                                    PROGMEM Tests -> Serial Port
* /                               Adrian Jones, February 2015
* /
/********************************************************************************************/

// Build 1
//   r1 150206 - initial build 

//********************************************************************************************
#define build 1
#define revision 1
//********************************************************************************************

#include <avr/pgmspace.h>

// Array of bytes (char) stored in PROGMEM
const char charBitmap [][8] PROGMEM = {              // custom characters
    { 0x00,0x00,0x00,0x00,0x00,0x07,0x07,0x07 },
    { 0x00,0x00,0x00,0x00,0x00,0x1C,0x1C,0x1C },
    { 0x00,0x00,0x00,0x00,0x00,0x1F,0x1F,0x1F },
    { 0x07,0x07,0x07,0x07,0x07,0x1F,0x1F,0x1F }, 
    { 0x1C,0x1C,0x1C,0x1C,0x1C,0x1F,0x1F,0x1F }, 
    { 0x1C,0x1C,0x1C,0x1C,0x1C,0x1C,0x1C,0x1C },
    { 0x07,0x07,0x07,0x07,0x07,0x07,0x07,0x07 },
    { 0x00,0x00,0x07,0x07,0x07,0x00,0x00,0x00 }
};

// Array of unsigned integers stored in PROGMEM
const unsigned int uintSet[]  PROGMEM = { 65535, 32796, 16843, 10, 11234};

// Array of character strings stored in PROGMEM
const char Line1  [] PROGMEM = "\tPROGMEM MEMORY TESTS";
const char Line2a [] PROGMEM = "\tFree Memory:  ";
const char Line2b [] PROGMEM = "B ";
const char *Lines [] PROGMEM = { Line1, Line2a, Line2b };

// Array of floats
const float fSet[] PROGMEM = {1.19278, 0.017452, 3.14159};

// Long piece of text
const char LongMes1[] PROGMEM  = {"The quick brown fox jumped over the lazy brown dog"};
const char LongMes2[] PROGMEM  = {"Never in the history of man has so much been owed by so many to so few"};

char cb[16];                 // character buffer for character strings. Set dimension to length of longest string
byte i;

//*****************************************************************************************//
//                                      Initial Setup
//*****************************************************************************************//
void setup() {

  Serial.begin(57600); 

  Serial.println(F("1. Accessing Character Strings"));
  strcpy_P(cb, (char*) pgm_read_word(&Lines[0] ) );       // read string into buffer
  Serial.println(cb);
  strcpy_P(cb, (char*) pgm_read_word(&Lines[1] ) );    
  Serial.print(cb);
  Serial.print(freeRam()); 
  strcpy_P(cb, (char*) pgm_read_word(&Lines[2] ) );    
  Serial.println(cb);

  Serial.println(F("2. Accessing Integers"));
  for (i = 0; i < (sizeof(uintSet)/sizeof(uintSet[0])); i++) {
    Serial.print(F("\t"));                                   
    Serial.print( i+1 );                                   
    Serial.print(F(".\t"));                                   
    Serial.println( pgm_read_word( &uintSet[i] ) );            // read a 2-byte integer
  }

  Serial.println(F("3. Accessing Floats"));
  for (i = 0; i < (sizeof(fSet)/sizeof(fSet[0])); i++) {
    Serial.print(F("\t"));                                   
    Serial.print( i+1 );                                   
    Serial.print(F(".\t"));                                   
    Serial.println( pgm_read_float( &fSet[i] ), 6 );                  // read a float
  }
  
  Serial.println(F("4. Accessing Byte Strings"));
  for (i = 0; i < (sizeof(charBitmap)/sizeof(charBitmap[0])); i++) {        // number of byte strings
    Serial.print(F("\t"));                                   
    Serial.print( i+1 );                                   
    Serial.print(F(".\t"));                                   
    for (byte j=0; j < (sizeof(charBitmap[0])/sizeof(charBitmap[0][0])); j++) {  // number of bytes in byte strings
      Serial.print(F("\t"));                                   
      Serial.print( pgm_read_byte( &charBitmap[i][j] ), HEX );                    // read each byte
    }
    Serial.println(F(""));
  }

    
/* Use this to create the 8 custom characters of an LCD display from byte strings in PROGMEM
  byte bb[8];                  // byte buffer for lcd custom characters
  for (byte i=0; i<8; i++ ) {                     // create 8 custom characters
    for (byte j=0; j<8; j++) bb[j]= pgm_read_byte( &charBitmap[i][j] );
    lcd.createChar ( i, bb );
  }
  */ 
  
  Serial.println(F("5. Accessing Long Strings and characters within strings"));
  Serial.print(F("\t1. '"));                                   
  for (i = 0; i < (sizeof(LongMes1)/sizeof(LongMes1[0]) - 1); i++) {        // number of byte strings
    Serial.print( char(pgm_read_byte_near(LongMes1 + i)) ); 
  }
  Serial.println(F("'"));
  Serial.print(F("\t2. '"));                                   
  for (i = 0; i < (sizeof(LongMes2)/sizeof(LongMes2[0]) - 1); i++) {        // number of byte strings
    Serial.print( char(pgm_read_byte_near(LongMes2 + i)) ); 
  }
  Serial.println(F("'"));
  Serial.print(F("\t   The 17th character of the second string is an \""));  
  Serial.print( char(pgm_read_byte_near(LongMes2 + 16)) ); 
  Serial.println(F("\"."));  

}

//*****************************************************************************************//
//                                      MAIN LOOP
//*****************************************************************************************//
void loop() {                   // run over and over again
}



// ********************************************************************************** //
//                                      OPERATION ROUTINES
// ********************************************************************************** //
// FREERAM: Returns the number of bytes currently free in RAM  
int freeRam(void) {
  extern int  __bss_end, *__brkval; 
  int free_memory; 
  if((int)__brkval == 0) {
    free_memory = ((int)&free_memory) - ((int)&__bss_end); 
  } 
  else {
    free_memory = ((int)&free_memory) - ((int)__brkval); 
  }
  return free_memory; 
}

[/codesyntax]

 

Enjoy saving SRAM !

3 thoughts on “Accessing Arduino Program Memory

  1. Pingback: Big Font LCD Characters | WoodUino.ca

  2. Gerard Venter

    Thanks for this detailed tutorial. It is starting to make sense. Compiled and run successfully on IDE 1.5.2. I tried on 1.6.5 and 1.6.6 with the update recommendations without success. It seems garbage is being written into PROGMEM, and so garbage coming out causes the serial monitor to only print the first character of the strings (1. A…). Seems apart from the const char * const issue there are also changes to the way Serial.println(F is dealt with. Were you successful on higher IDEs ?

    Reply
    1. Adrian Post author

      Gerard,
      I don’t know if you got your code working but if not, please post your code here and I’ll see if I can assist.
      … Adrian

      Reply

Come on... leave a comment...