NEW: Learning electronics? Ask your questions on the new Electronics Questions & Answers site hosted by CircuitLab.
Everything Else » C program to read and write raw data on SD card
January 07, 2013 by scootergarrett |
I would like to be able to read and write the raw data from an SD card using just a computer (no microprocessor) forgetting the FAT32. I would like to do this in basic C. I can look at the raw data with a program called winhex. Long run I want someone who is less computer savvy to run a C script that will store certain parameters on SD card (where I say so) that the microprocessor would use. Then store data on it allowing me to never have to connect directly to a computer. I’m having a hard time getting started with this program partly because I’m not even sure what to search for (I keep getting sights trying to sell me stuff). So if anyone has knowledge of the CreateFile() and ReadFile() you help would be great. Here is my code so far with problem area commented (Windows XP, CodeBolcks10.05) mostly from some people at cprogramming.com
So it’s (I'm) having a hard time reading data. I like trouble shooting programming problems by breaking code up but I don’t even know where to start with this problem, any ideas? Thanks |
---|---|
January 07, 2013 by Noter |
To access raw data outside a file system you have to use ioctl's which are low level io calls that get passed directly to device drivers. Typically information is passed in the ioctl with pointers to various system structures and likewise returned. It's a little tougher than regular application level programming. Seems to me it would be easier to go ahead and use FAT to keep the PC programming simpler and it won't be too hard on the AVR side either because it's been done many times and there are several open source libraries on the net. |
January 07, 2013 by Noter |
Actually I'm not so sure about having to use ioctls to read/write block devices. But probably you should start by opening the device as input instead of create and just get reading to work before moving on to writing/formatting. If you can write it you can zero/format it yourself |
January 07, 2013 by Noter |
This is for linux but windows will be similar - take a look at chapter 6 and appendix B. |
January 08, 2013 by scootergarrett |
You are probably right with just using FAT32, but I would just like to be able to get all 8GB from my 8GB card, not 7.6GB. just something I would like to be able to do. I will take a look at the linux stuff when I get a chance, thanks for the lead. |
January 08, 2013 by pcbolt |
Garret - On Windows, I'm pretty sure you need to be in Kernel mode to get the kind of access you're talking about. That means having to write a driver for your "User" mode program to go through to access the SD card. That's a tall order. Oddly enough, using the MCU is easy. You can hook up an SD card to it on the SPI lines and just pass through PC commands/data on the UART Tx/Rx lines. There are some links to more information HERE. |
January 08, 2013 by Noter |
Open Source examples, rawcopy.cpp is probably closest to what you are trying to do. |
January 08, 2013 by Noter |
I would still go with FAT and just use a 16/32/64gb card if I needed the extra .4gb storage. Never know, might want to read that card from your cell phone or whatever some day and they all like file systems. But on the other hand, it is fun to be able to do exactly what you want and I always enjoyed learning/using the less common (and more difficult) programming techniques. |
January 08, 2013 by scootergarrett |
Yes I’m on the side of “if I spend 10 dollars on a SD card I want to be able to use it how I want to”. I don’t let the man tell me how to use it. Thanks for the ‘less common more difficult programming techniques’ that are project specific better. |
January 10, 2013 by pcbolt |
Noter - Were you able to extract a rawcopy.cpp from that site? All I got was an .exe (no source code). Garret - I think the .4gb is allocated for the FAT32 file system expansion itself. Not much to see there unless you want to explore how things are organized. Most of it is blank. |
January 10, 2013 by JimFrederickson |
There is no real mystery to this. The DOS Function calls are still available as well as several methods from within Windows itself. You do, of course, have to run with Administrator rights unless you are using a wrapper/library that takes care of the Security for you. If you are writing directly to storage devices you will have to be very careful, which I am sure that you have considered. A small error in your program could crash/corrupt a legitimate file system that you would prefer to be left intact. (Your boot drive for instance.) Misswriting a single cluster, even sector, can have essentially instantaneous and disastrous results. Method #1:
Method #2:
I think that those 2 examples should be enough for you to find out more. |
January 10, 2013 by JimFrederickson |
I would advice, at the very least, to use a partition table that Windows would recognize... |
January 10, 2013 by Noter |
pcbolt - The link for sources is at the top of the page. |
January 10, 2013 by scootergarrett |
Thanks so much Jim I’ve got it now. And I didn’t corrupt my hard drive doing it. I had to change your code a little to get to work on my compiler/ computer (I’m not good enough with C to get into compilers). I noticed that the first sector of the SD card has important data and needs to be left alone. If it’s written over it won’t let you open it next time and a re-format is necessary, I’m sure there is a simple explanation. I can’t wait to use all 8GB for data. Thanks again. This to write to a sector
And this to read from a sector
|
January 10, 2013 by pcbolt |
Jim - Thanks for that code. Learned something new everyday here. |
January 10, 2013 by JimFrederickson |
That is good nothing got corrupted. People often times see REALLY LARGE HARD DRIVES and don't really realize that a very small amount of data misswritten and it could all be gone... (Well "technically", not gone but rather inaccessible...) If you go to Wikipedia and look up "Fat32" it will redirect you to "File Allocation Table". There you will find a good technical description on what the "First Sector" is all about. (That was my previous remark about a Partition Table. It is just easier to keep one that Windows would recognize even if the File System is invalid...) I usually find it easier, more helpful, to have some sort of file system, but there are times where that isn't really important. Especially since small flash cards are REALLY CHEAP and still provide HUGE STORAGE for a Microcontroller. Another thing you can do, which you may have thought of, is getting a few of the MicroSD Cards. You can use the MicroSD-to-SD-Card-Adapter as your Flash Socket and solder directly onto the end of it. (I find that helpful/convenient too!) |
January 10, 2013 by JimFrederickson |
Glad the code could get in you the proper direction and or is of interest. (To "Both"...) |
January 16, 2013 by scootergarrett |
Now I’m having another problem, when I use the fwrite function if one byte in the buffer is 0x0A (10) it won’t load it correctly. I think its treating this as “NL line feed, newline”. Any ideas how to make fwrite just print the data in the buffer exactly how it is. Thanks. |
January 16, 2013 by scootergarrett |
OK I figured it out here it all about “ab” opens it in binary form.
|
January 16, 2013 by Noter |
SG, have you considered using CLKO to provide an external clock for your slave chip? Even the same freq crystals will drift a bit over time but if both/all chips run off the same clock they will stay in step forever. |
January 16, 2013 by scootergarrett |
I think that post was intended for this thread. Anyways my overall idea is to have 2 completely independent sensors not connected at all. But as I just learned crystals have a stability rating in ppm (parts per million isn’t a just a chemistry thing anymore) and with an error of 100ppm there will be an error of 4.3 minutes per month. My expected run time is about 3 hours resulting in 1.08 sec shift at worst case, which I might just have to live with. But here is a concept I just thought up, what if there was some way to re connect them after and detect the phase shift. That sounds too complicated for now. |
January 17, 2013 by JimFrederickson |
Scooter, I am curious, what C Compiler are you using for your Windows Programming? |
January 17, 2013 by scootergarrett |
I have CodeBlocks 10.05 I don’t know much about picking compilers but it says ‘GNU GCC’ under the compiler setting so I stick to that. |
January 23, 2013 by Ralphxyz |
Wow scootergarrett, Code::Blocks is a fantastic IDE. It even has the AVR GCC compiler all set to build AVR projects besides doing C and C++. I am just starting my first attempt so I assume there will be a learning curve, but so far it seems pretty straight forward. Thanks so much. Ralph |
March 28, 2013 by scootergarrett |
New micro-SD project is chilling out the window talking temperature data. So I have a set up so I can put a sampling frequency and duration on an SD card pop it in the MCU turn it on and let it run. Then take the card out and extract the data. Data to and from the nerdkit with no wires. The code defiantly needs work, if the duration is set to long bad things are happening. I think the writing to the SD card is causing it to skip AD conversions but if I better work out my interrupts I can probably fix that (testing soon). Anyways I have the whole circuit on 3.3 Volts to better accommodate the SD card. Powered by a little 6V Duracell 28L which might be my new battery of choice for small projects that still need to last (250mAhr also needs testing). Circuit diagram Circuit lab. And MC code so far:
Will post the data extract code if anyone cares Out window plot running a few minutes Check out that first order response |
March 30, 2013 by pcbolt |
SG - I see where you set the frequency
but I didn't see where the duration is set. I do know the writes to the SDHC card are pretty slow (in MCU time) so you could try increasing the SPI clock frequency or maybe just disable the timer interrupt while your writing to the card. You can also set the ADC to operate in "Auto Trigger Mode" when the timer hits OCR1A value, then have the the ADC "on complete" interrupt store the results and update the buffer pointer. Then all your "main" code has to do is check the buffer pointer (if it's a global variable) and write to the card when it's full. If you're worried about battery life you can also look into the various sleep modes...but I guess one thing at a time, eh? |
April 07, 2013 by scootergarrett |
The durations is set by data already on the SD card. When I initialize the SD card I determine the Last sector that will be used, so then I just need to break out of the cycle if the current sector is == to the last sector. So for running faster I changed the code so the buffer is getting fill constantly every interrupt and the SD card is getting written when the buffer is full, so as long as the SD card get the first couple of numbers in the buffer before the next interrupt starts filling the buffer again there wont be a problem. One issue now is that my computer can’t malloc data for 11520000 points (8hr at 400 Hz). It will statically allocate that much memory but then numbers with in the program need to be change, I like the ability to pop the card in and have the program malloc the correct amount of data (thinking of non computer people using the sensor). I also added a sleep mode to the end so when the sampling time is over it will save battery life. CODE:
So when I moved to taking 2 measurements back to back I got some strange results. This is with 2 identical temperature sensors. There is one bit noise half the time it takes to fill a buffer, any ideas what could cause this? Could communicating with the SD card be affecting measurements Interrupt code for 2 measurements at a time:
|
Please log in to post a reply.
Did you know that our USB NerdKit works on Windows, Linux, and Mac OS X? Learn more...
|