Recent Changes - Search:
Project /

GPSLogger

NAVIGATOR DATA LOGGER

GPS Logger Development Plan/Checklist

References

http://www.piclist.com/images/ORG/elm-chan/http/docs/mmc/mmc_e.html

W I S H L I S T

2008-02-18

Introduction / Purpose

These are random thoughts I've had over the past few weeks regarding the design and construction of a small electronic device that would capture the serial data output from my airplane's navigation radio (GPS receiver), simply capturing that data for future use. These uses might be:

1- Enjoying a 'replay' of the flight on a moving map system (FliteMap) 2- Providing a historical record of flights taken and routes traveled. 3- Serve as defensible evidence in case of 'official' investigation of claimed penetration of restricted and prohibited airspace

General Characteristics

Fundamental Expectations

I want the data logger to be a small, very simple to use, self-contained, portable device that interfaces with the serial data output available on most LORAN and GPS navigation devices. In my mind I see something about the size of a portable MP3 player.

I'd like the storage medium to be some form of flash memory card that is portable between the data logger device and a personal computer. I'm hoping this can be an 'Secure Digital' (SD) type of card. MultiMedia Card (MMC) flash devices might also serve, but they have been obsolted pretty much by SD. Most computers that have any form flash memory card reader will support SD.

The data logger device should operate on battery power, with many hours of use before requiring battery replacement. I'd like to use the smallest (physical size) batteries possible, say, two, three, or four AAA cells, or if possible, a single, or double set of lithium coin cells, say the CR2032.

I'd like the device to be as easy to use as possible, in the sense that it should detect the initiation of data flow and power itself up. Likewise, it should detect the loss of data flow and after an appropriate amount of time has elapsed, turn itself off. These two capabilities make the device fully 'hands-off' during normal flight operations.

The device will need some mechanism for interfacing with the user. I can imagine a couple of ways this can be done:

  1. - It might be as simple as a POWER LED, possibly with a DATA DETECTED LED. These at a minimum would give sufficient indication to the user that the data logger is awake and doing its job.
  2. - A small general purpose LCD screen can be used that shows memory card capacity, battery status, and other interesting data. This will cost a lot more money then the LEDs, of course.
  3. - The device should probably have, at the very least, a power button that can be used to force the unit on and off.
Nice-To-Haves

It would be very nice for the device to have USB connectivity, so that it can be directly connected to a personal computer and data downloaded without having to remove the flash memory card. This, though, presumably creates a whole new set of interface headaches, and may require a good deal more progamming code than makes sense.

It would be nice for the data stored in the flash memory device to be file structured, allowing for copying and use of specific data logs, say, for a particular flight, or day. At this point, it is unclear to me how that is done. For example, my digital cameras store pictures in individual named files, and my Palm PDA stores files and MP3s in named files on SD cards. I guess there is a software layer between the camera and the flash card that IS a file system. Research suggests that most devices that use flash memory use some kind of FAT16 or FAT32 compatible / equivelant file system.

Resources

Over the past two days I've done some research to educate myself on flash devices and microcontrollers. I've found several useful websites:

In fact, the first URL above includes a schematic and code for just such a data logger. The information given is very generic, so I would need to make many enhancements, but it looks like a very useful resource for starting out.

The last URL, which isn't necessarily directly useful to my project, gives interesting information that is useful background material regarding NAND flash devices. Of particular note, is the fact that NAND memory devices are block devices, and don't allow random reads and writes to any single byte of data. This will clearly influence the eventual software design.

Considerations

After a little research into flash file systems (YAFFS, JFFS2) and which itself prompted some self-education on the 'wear-leveling' concept, I'm now of the opinion that the data logger files be simply stored in a RAW format of my own design. The overhead for YAFFS appears to be on the order of 10-15K of code, which is certainly not available on a PIC microcontroller type processor. This idea of having a PC-readable filesystem will have to wait for another iteration, or if this device ever goes from a personal prototype to a production product.

If I simply store the serial data stream sequentially on the flash device, then I'll be alright. It can be as simple as having some kind of data file catalog that has an entry for each file, with starting (and ending) block numbers. Of course, this means that if the card is to be read on a PC, then I'll have to write some kind of driver for that platform. Ugh! Or, the data is transfered from the device to a PC over the serial link, which seems very simple, in contrast.

If a simple sequential storage scheme is used, then it makes sense, operationally, to just let the memory fill up, more or less, then erase the whole flash ans start again. This minimizes wear-leveling issues (which I'm not doing any specific about) and makes for a simple 'catalog' file structure. I would just set aside so many blocks for the catalog, and the rest of the memory is used to hold data. If the file is truncated, because of loss of power or the system powers off after the end of a flight, or whatever, then to start a new file I just search for the from the beginning of the data area for the first 'blank' block, and use that block as the start of the next file, updating the catalog. Pretty simple, I think.

August 15, 2008

Parts list

Using "Captain's" PIC circuit as a model, here are the parts that are required:

  • (1) PIC16F876 PIC16F886
  • (1) MAX232 MAX242 (low-power mode)
  • (1) 78L05 L4931 5.0v
  • (1) LD1086 L4931 3.3v
  • (1) 4MHz Osc.
  • (3) 100nF (.1uF) cap
  • (4) 1uF cap
  • (2) 10uF cap
  • (3) 1k8 resistor
  • (3) 3k3 resistor
  • (1) MMC/SD card socket
  • (1) 24 PDIP socket 28-pin socket
  • (1) 14 PDIP socket 18-pin socket
  • (1) Veroboard/breadboard

Found this site for simplified programming of all the PIC and AVR chips: http://gcbasic.sourceforge.net/

2018-02-05

I've been thinking more about this project, since getting a PIC Projects book for Christmas. The PIC chips that I got oh so long ago are the PIC16F886, which is a very capable 28-PIN skinny DIP chip. It has internal oscillator, 384 bytes of RAM, 8K program flash memeory, UART, comparators, GPIO ports, low power mode, sleep mode, and a bunch of other stuff. I'm convinced that it would be a good device for the heart of the project.

Last night I searched the web and found quite a bit of information about interfacing PICs with SDcards. I think that would be the way to go. Interfacing with the SDcard uses only three pins and is easy to drive with a PIC. The chief issue, which is now understood, is that I wanted to be able to remove the SDcard from the device and read the data on a desktop computer. The issue is this: a FAT filesystem would be the way to go, but the fat libraries that I found for the PIC are designed for the larger 16-bit PIC18xxxxx chips. They typically have 512 to 2K bytes of RAM which allows for whole sector reading and writing. Not only that, at the filesystem level, but the SDcard is also read and written in 512-byte 'sectors' or blocks, too. My PIC16 cannot do that... BUT... I think I have found a way.

My intent all along has been to design the system in such a way as to simply gather the NMEA-0183 sentences that are of interest from the GPS unit(s) and store it in the device for later off-line processing. That's it. Now, it might be best to simply have the stream of ASCII data flow from the GPS into a file on the logger, but since I can't buffer a whole 512-byte block, actual FAT filesystem operations are not an options. Okay. But, there is no reason that a properly pre-formated SDcard with one single NULL-filled file that occupies the entire SDcard couldn't exist, then, as NMEA-0183 sentences come in I simply write each sentence, NULL-terminated, to it's own sequential block, starting with the first block that is 'data' in the SDcard's FAT filesystem. Now the file just grows as data comes in, and later, I can take the SDcard to another standard PC and simply read the data out. If I want, I can also delete the NULLs as I encounter them. This would leave me with the data, usable and all, but without having the logger do any filesystem stuff. Pretty cute, eh?

Sure, there will be tons of 'wasted' space on the SDcard, but it doesn't matter. Let's to a little math:

  • each NMEA-0183 sentence will be no more than 83 bytes long. (Let's use 80 as a typical value)
  • a 1GB SDcard has 2^21 512-byte sectors/blocks, that's 2,097,152 sectors.
  • if we record one sentence a second:
    • 2,097,152 / 24 (hours per day) / 60 (minutes per hour) / 60 (seconds)
    • we get over 24 days of logging capacity ! ! !
  • That's 24 hours a day for more than 24 days!

I would say that the inefficiency of data storage, in leaving so much space -- roughly 85% of each sector -- unused is not at all troublesome. There is plenty of capacity for our purposes.

And there is a side benefit. Because we are not actually performing any filesystem functions while we log the data, we are not updating the FAT directory or tables (sector bitmaps) -- we will simply be writing to each next sector sequentially, there is no worry about wear-balancing on the flash device. It simply isn't a problem. The only key requirement is that we somehow keep a record, either on the PIC chip itself, or in the SDcard of which sector/block is next in sequence.

I can see several ways of doing that. Of course the PIC will have the NEXT-SECTOR count in internal memory while in operation, but we need to store it when we shutdown. That could be:

  • When data stops flowing, a watchdog process notices, and before device shutdown, calls a CleanUp routine which:
    1. writes the NEXT-SECTOR value into the PIC's internal flash RAM area.
    2. writes the NEXT-SECTOR value into an unused (last?, or the MBR?) sector on the SDcard.
    3. writes a whole block of NULLs into the next sector
  • Each of these would require at startup:
    1. read NEXT-SECTOR from internal flash RAM. The hazard here is that if the SDcard is swapped out while off-line, NEXT-SECTOR is no longer valid, and could result in lost data on the new card.
    2. read NEXT-SECTOR from the defined location in the SDcard. This eliminates the foregoing problem.
    3. scan the device until it locates the first NULL-filled sector, which marks the next sector. This might take a long time though, delaying logger startup to the point that it isn't useful. The scan would have to occur in a matter of seconds, five or less, to be fair on the user.