OpenLog Hookup Guide a learn.sparkfun.com tutorial

Available online at: http://sfe.io/t498

Contents

Introduction

Heads up! This tutorial is for the Open Log for serial UART [ DEV-13712 ]. If you are using the Qwiic OpenLog for I2C [ DEV-15164 ], please refer to the Qwiic OpenLog Hookup Guide.

The OpenLog Data Logger is a simple-to-use, open-source solution for logging serial data from your projects. The OpenLog provides a simple serial interface to log data from a project to a microSD card.

SparkFun OpenLog

SparkFun OpenLog

DEV-13712
$16.95
22
SparkFun OpenLog with Headers

SparkFun OpenLog with Headers

DEV-13955
$17.50
3
no product found

Materials Required

In order to fully work through this tutorial, you will need the following parts. You may not need everything though depending on what you have. Add it to your cart, read through the guide, and adjust the cart as necessary.

Recommended Reading

If you are not familiar or comfortable with the following concepts, we recommend reading through these before continuing on with the OpenLog Hookup Guide.

How to Solder: Through-Hole Soldering

This tutorial covers everything you need to know about through-hole soldering.

Serial Communication

Asynchronous serial communication concepts: packets, signal levels, baud rates, UARTs and more!

Serial Peripheral Interface (SPI)

SPI is commonly used to connect microcontrollers to peripherals such as sensors, shift registers, and SD cards.

Serial Terminal Basics

This tutorial will show you how to communicate with your serial devices using a variety of terminal emulator applications.

Hardware Overview

Power

The OpenLog runs at the following settings:

OpenLog Power Ratings
VCC Input3.3V-12V (Recommended 3.3V-5V)
RXI Input2.0V-3.8V
TXO Output3.3V
Idle Current Draw~2mA-5mA (w/out microSD card), ~5mA-6mA (w/ microSD card)
Active Writing Current Draw~20-23mA (w/ microSD card)


The OpenLog's current draw is about 20mA to 23mA when writing to a microSD. Depending on the size of the microSD card and its manufacturer, the active current draw can vary when the OpenLog is writing to the memory card. Increasing the baud rate will also pull more current.

Microcontroller

The OpenLog runs off of an onboard ATmega328, running at 16MHz thanks to the onboard crystal. The ATmega328 has the Optiboot bootloader loaded on it, which allows the OpenLog to be compatible with the "Arduino Uno" board setting in the Arduino IDE.

ATmega328

The brain of the OpenLog.

Interface

Serial UART

The primary interface with the OpenLog is the FTDI header on the board edge. This header is designed to plug directly into an Arduino Pro or Pro Mini, which allows the microcontroller to send the data over a serial connection to the OpenLog.

FTDI_Header

Serial Connection Header.

SPI

There are also four SPI test points broken out on the opposite end of the board. You can use these to reprogram the bootloader on the ATmega328.

SPI Test Points

SPI Test Points.

The latest OpenLog (DEV-13712) breaks out these pins on smaller plated through holes. If you need to use an ISP to reprogram or upload a new bootloader to the OpenLog, you can use pogo pins to connect to these test points.

The final interface for communicating with the OpenLog is the microSD card itself. To communicate, the microSD card requires SPI pins. Not only is this where the data is stored by the OpenLog, but you can also update the OpenLog's configuration via the config.txt file on the microSD card.

microSD Card

All data logged by the OpenLog is stored on the microSD card. The OpenLog works with microSD cards that involve the following features:

microSD Slot

microSD Slot on the bottom of the OpenLog.

Status LED

There are two status LEDs on the OpenLog to help you with troubleshooting.

Stat LEDs

Status LEDs on the OpenLog.

Hardware Hookup

There are two main methods for connecting your OpenLog to a circuit. You will need some headers or wires to connect. Make sure that you solder to the board for a secure connection.

Basic Serial Connection

This hardware connection is designed for interfacing with an OpenLog if you need to reprogram the board, or log data over a basic serial connection.

Make the following connections:

OpenLog → 3.3V FTDI Basic Breakout

Notice that it is not a direct connection between the FTDI and OpenLog - you must switch the TXO and RXI pin connections.

Your connections should look like the following:

OpenLog + FTDI

Once you have the connections between the OpenLog and the FTDI Basic, plug your FTDI board into a USB cable and into your computer.

Open up a serial terminal, connect to the COM port of your FTDI Basic, and go to town!

Project Hardware Connection

While interfacing with the OpenLog over a serial connection is important for reprogramming or debugging, the place where OpenLog shines is in an embedded project. This general circuit is how we recommend you hook up your OpenLog to a microcontroller (in this case, an Arduino Pro Mini) that will write serial data out to the OpenLog.

First you will need to upload the code to your Pro Mini you intend to run. Please check out the Arduino Sketches for some example code that you can use.

Once you have programmed your Pro Mini, you can remove the FTDI board, and replace it with the OpenLog. Make sure to connect the pins labeled BLK on both the Pro Mini and OpenLog (the pins labeled GRN on both will also match up if done correctly).

If you cannot plug the OpenLog directly into the Pro Mini (due to mismatched headers or other boards in the way), you can use jumper wires and make the following connections.

OpenLog → Arduino Pro/Arduino Pro Mini

Once you're finished, your connections should look like the following with the Arduino Pro Mini and Arduino Pro. The Fritzing diagram shows the OpenLogs with the headers mirrored. If you flip the microSD socket relative to the Arduino's top view, they should match the programming header like an FTDI.

OpenLlog to Arduino Pro Mini OpenLog to Arduino Pro

Note that the connection is a straight shot with the OpenLog "upside-down" (with the microSD facing up).

Power up your system, and you are ready to start logging!

Arduino Sketches

There are six different examples sketches included that you can use on the Arduino when connected to an OpenLog.

Firmware

The OpenLog has two primary pieces of software on board: the bootloader and the firmware.

Arduino Bootloader

Note: If you are using an OpenLog that was purchased prior to March 2012, the onboard bootloader is compatible with the "Arduino Pro or Pro Mini 5V/16MHz w/ ATmega328" setting in the Arduino IDE.

As mentioned previously, the OpenLog has the Optiboot serial bootloader on board. You can treat the OpenLog just like an Arduino Uno when uploading example code or new firmware to the board.

If you end up bricking your OpenLog and need to reinstall the bootloader, you will also want to upload Optiboot onto the board. Please check out our tutorial on installing an Arduino Bootloader for more information.

Compiling and Loading Firmware onto the OpenLog

Note: If this is your first time using Arduino, please review our tutorial on installing the Arduino IDE. If you have not previously installed an Arduino library, please check out our installation guide to manually install the libraries.

If for any reason you need to update or reinstall the firmware on your OpenLog, the following process will get your board up and running.

First, please download the Arduino IDE v1.6.5. Other versions of the IDE may work to compile the OpenLog firmware, but we have verified this as a known good version.

Next, download the OpenLog firmware and required libraries bundle.

Download OpenLog Firmware Bundle (ZIP)

Once you have the libraries and firmware downloaded, install the libraries into Arduino. If you are unsure how to manually install the libraries in the IDE, please check out our tutorial: Installing an Arduino Library: Manually Installing a Library.

Note: We are using modified versions of the SdFat and SerialPort libraries in order to arbitrarily declare how big the TX and RX buffers should be. The OpenLog requires the TX buffer to be very small (0) and the RX buffer needs to be as large as possible. Using these two modified libraries together allows increased performance of the OpenLog.

Next, to take advantage of the modified libraries, modify the SerialPort.h file found in \Arduino\Libraries\SerialPort directory. Change BUFFERED_TX to 0 and ENABLE_RX_ERROR_CHECKING to 0. Save the file, and open up the Arduino IDE.

If you haven't yet, connect your OpenLog to the computer via an FTDI board. Please double check the example circuit if you are not sure how to do this properly.

Open the OpenLog sketch you would like to upload under Tools>Board menu, select the "Arduino/Genuino Uno", and select the proper COM port for your FTDI board under Tools>Port.

Upload the code.

That's it! Your OpenLog is now programmed with new firmware. You can now open up a serial monitor and interact with the OpenLog. On power up, you will see either 12> or 12<. 1 indicates the serial connection is established, 2 indicates the SD card has successfully initialized, < indicates OpenLog is ready to log any received serial data and > indicates OpenLog is ready to receive commands.

OpenLog Firmware Sketches

There are three included sketches you can use on the OpenLog, depending on your particular application.

Command Set

You can interface with the OpenLog via a serial terminal. The following commands will help you read, write, and delete files, as well as change the settings of the OpenLog. You will need to be in Command Mode in order to use the following settings.

While the OpenLog is in Command Mode, STAT1 will toggle on/off for every character received. The LED will stay on until the next character is received.

File Manipulation

Directory Manipulation

Low Level Function Commands

System Settings

These settings can be manually updated, or edited in the config.txt file.

Note: During normal logging, echo will be turned off. The system resource demands for echoing the received data is too high during logging.
Remember: If you get the board stuck in an unknown baud rate, you can tie RX to GND and power up OpenLog. The LEDs will blink back and forth for 2 seconds and will then blink in unison. Power down the OpenLog, and remove the jumper. OpenLog is now reset to 9600bps with an escape character of `CTRL-Z` pressed three consecutive times. This feature can be overridden by setting the Emergency Override bit to 1. See config.txt for more info.

Configuration File

If you would rather not use the serial terminal for modifying the settings on your OpenLog, you can also update the settings by modifying the CONFIG.TXT file.

Note: This feature only functions on firmware verison 1.6 or newer. If you have bought an OpenLog after 2012, you will be running firmware version 1.6+

To do this, you will need a microSD card reader and a text editor. Open up the config.txt file (the capitalization of the file name does not matter), and configure away! If you have never powered up your OpenLog with the SD card before, you can also manually create the file. If you have powered up the OpenLog with the microSD card inserted previously, you should see something like the following when you read the microSD card.

Default SD Configuration

The OpenLog creates a config.txt and LOG0000.txt file on first power up.

The default configuration file has one line of settings and one line of definitions.

OpenLog Config File

Default configuration file written by the OpenLog.

Note that these are regular visible characters (there are no non-visible or binary values), and each value is separated by a comma.

The settings are defined as follows:

How OpenLog Modifies the Config File

There are five different situations for the OpenLog to modify the config.txt file.

Troubleshooting

There are several different options to check if you are having issues connecting over the serial monitor, having issues with dropped characters in logs, or fighting a bricked OpenLog.

Check STAT1 LED Behavior

STAT1 LED shows different behavior for two different common errors.

Double Check Subdirectory Structure

If you are using the default OpenLog.ino example, OpenLog will only support two subdirectories. You will need to change FOLDER_TRACK_DEPTH from 2 to the number of subdirectories you need to support. Once you've done this, recompile the code up, and upload the modified firmware.

Verify the Number of Files in the Root Directory

OpenLog will only support up to 65,534 log files in the root directory. We recommend reformatting your microSD card to improve logging speed.

Verify the Size of your Modified Firmware

If you are writing a custom sketch for the OpenLog, verify that your sketch is not larger than 32,256. If so, it will cut into the upper 500 bytes of Flash memory, which is used by the Optiboot serial bootloader.

Double Check File Names

All file names should be alpha-numeric. MyLOG1.txt is ok, but Hi !e _.txt may not work.

Use 9600 Baud

OpenLog runs off of the ATmega328 and has a limited amount of RAM(2048 bytes). When you send serial characters to OpenLog, these characters get buffered. The SD Group Simplified Specification allows an SD card to take up to 250 ms (section 4.6.2.2 Write) to record a data block to flash memory.

At 9600 bps, that's 960 bytes (10 bits per byte) per second. That is 1.04 ms per byte. OpenLog currently uses a 512 byte receive buffer so it can buffer around 500 ms of characters. This allows OpenLog to successfully receive all characters coming at 9600 bps. As you increase the baud rate, the buffer will last for less time.

OpenLog Buffer Overrun Time
Baud RateTime per byteTime Until Buffer is Overrun
9600 bps1.04 ms532 ms
57600 bps0.174 ms88 ms
115200 bps0.087 ms44 ms

Format your MicroSD Card

Remember to use a card with few or no files on it. A microSD card with 3.1GB worth of ZIP files or MP3s has a slower response time than an empty card.

If you did not format your microSD card on a Windows OS, reformat the microSD card and create a DOS filesystem on the SD card.

Swap MicroSD Cards

There are many different types of card manufacturers, relabeled cards, card sizes, and card classes, and they may not all work properly. We typically use an 8GB class 4 microSD card, which works well at 9600bps. If you need higher baud rates, or larger storage space, you may want to try class 6 or above cards.

Add Delays Between Character Writes

By adding a small delay between Serial.print() statements, you can give OpenLog a chance to record its current buffer.

For example:

language:c
Serial.begin(115200);      
for(int i = 1 ; i < 10 ; i++) {
    Serial.print(i, DEC);     
    Serial.println(":abcdefghijklmnopqrstuvwxyz-!#");
}

may not log properly, as there are a lot of characters being sent right next to each other. Inserting a small delay of 15ms between large character writes will help OpenLog record without dropping characters.

language:c
Serial.begin(115200);      
for(int i = 1 ; i < 10 ; i++) {
    Serial.print(i, DEC);     
    Serial.println(":abcdefghijklmnopqrstuvwxyz-!#");
    delay(15);
}

Add Arduino Serial Monitor Compatibility

If you are attempting to use the OpenLog with the built-in serial library or the SoftwareSerial library, you may notice issues with command mode. Serial.println() sends both newline AND carriage return. There are two alternative commands to overcome this.

The first is to use the \r command (ASCII carriage return):

language:c
Serial.print("TEXT\r");

Alternatively, you can send the value 13 (decimal carriage return):

language:c
Serial.print("TEXT");
Serial.write(13);

Emergency Reset

Remember, if you need to reset the OpenLog to a default state, you can reset the board by tying the RX pin to GND, powering up the OpenLog, waiting until the LEDs begin to blink in unison, and then powering down the OpenLog and removing the jumper.

If you have changed the Emergency Override bit to 1, you will need to modify the configuration file, as the Emergency Reset will not work.

Check with the Community

If you are still having issues with your OpenLog, please check out the current and closed issues on our GitHub repository here. There is a large community working with the OpenLog, so chances are that someone has found a fix for the problem you are seeing.

Resources and Going Further

Now that you've successfully logged data with your OpenLog, you can set up remote projects and monitor all the possible data coming. Consider creating your own Citizen Science project, or even a pet tracker to see what Fluffy does when out and about!

Check out these additional resources for troubleshooting, help, or inspiration for your next project.

Need some more inspiration? Check out some of these related tutorials:

MicroSD Breakout With Level Shifter Hookup Guide

How to use the level translating µSD device.

micro:climate Kit Experiment Guide

A weather station kit that is built on top of the inexpensive, easy-to-use micro:bit and Microsoft MakeCode.

SparkFun RTK Facet Hookup Guide

Setup the RTK Facet in minutes to begin gathering millimeter level geospatial coordinates.

Qwiic Kit for Raspberry Pi V2 Hookup Guide

Get started with the SGP40, BME280, VCNL4040, and microOLED via I2C using the Qwiic system and Python on a Raspberry Pi! Measure VOC Index, light, temperature, humidity, and pressure from the environment. Then display them on the microOLED, serial terminal, or the cloud with Cayenne!

Or check out these blog posts for ideas.

On the Origin of Thermal Sensors

March 3, 2015

Data Logging for Air Quality

December 2, 2020

If you have any tutorial feedback, please visit the comments or contact our technical support team at TechSupport@sparkfun.com.


learn.sparkfun.com | CC BY-SA 3.0 | SparkFun Electronics | Niwot, Colorado