FatFs on ESP32 with esp-idf can't create file with *.json extension - fatfs

I want to switch from SPIFFS to FAT on my Esp32 projects due to encryption. In my example project I have this.
esp_vfs_spiffs_conf_t conf = {
.base_path = "/spiffs",
.partition_label = NULL,
.max_files = 5, // This decides the maximum number of files that can be created on the storage
.format_if_mount_failed = true
};
esp_err_t ret = esp_vfs_spiffs_register(&conf);
...
ESP_LOGI(TAG, "Opening file 1");
FILE *f = fopen("/spiffs/hello.json", "w");
if (f == NULL) {
ESP_LOGE(TAG, "Failed to open file 1 for writing test");
}
ESP_LOGI(TAG, "Opening file 2");
FILE *f2 = fopen("/spiffs/hello.txt", "w");
if (f2 == NULL) {
ESP_LOGE(TAG, "Failed to open file 2 for writing test");
return ESP_FAIL;
}
It works fine and creates both files as expected, but this:
esp_vfs_fat_mount_config_t conf = {
.format_if_mount_failed = true,
.max_files = 5,
.allocation_unit_size = CONFIG_WL_SECTOR_SIZE
};
esp_err_t ret = esp_vfs_fat_spiflash_mount("/fatfs", "storage", &conf, &s_wl_handle);
ESP_LOGI(TAG, "Opening file 1");
FILE *f = fopen("/fatfs/hello.json", "w");
if (f == NULL) {
ESP_LOGE(TAG, "Failed to open file 1 for writing test");
}
ESP_LOGI(TAG, "Opening file 2");
FILE *f2 = fopen("/fatfs/hello.txt", "w");
if (f2 == NULL) {
ESP_LOGE(TAG, "Failed to open file 2 for writing test");
return ESP_FAIL;
}
fails for the json file.
I (3672) example: Opening file 1
E (3672) example: Failed to open file 1 for writing test
I (3682) example: Opening file 2
...
Google so far gave me nothing on FatFS having any issue with file extensions. Can someone help me understand this?

I hope the issue is already resolved, but I will answer anyway to support other developers with a similar struggle.
I believe this may happen because you didn't enable long file names for FATFS. With no LFN support, you only can have three characters in the extension (and 8 in the filename). To fix this, please invoke menuconfig using idf.py menuconfig and then head to Component config -> FAT Filesystem support -> Long filename support -> Long filename buffer {in heap|on stack}.
Then rebuild the project and check if it works now.

Related

Where is the default path to read config or data file in ROS when using "roslaunch" to launch the nodes?

In a project , I have to use an .so file , and the .so need to load a config file that I provide. The problem is that I can not change the code of .so , but the .so load the config file in default path.
When I use rosrun to launch the node , I put the config file in the catkin workspace ,and when I use ./ to launch the node, I put the config file under the same folder with node. My test program is also the result.
My test code:
int main(int argc, char **argv)
{
ros::init(argc, argv, "topic_pub");
ros::NodeHandle nh;
ros::Publisher msg_pub = nh.advertise<rostestdemo::NewMsg>("customize_msg", 20);
ros::Rate loop_rate(10);
rostestdemo::NewMsg msg;
int count = 0;
std::string filename = "hello.txt";
std::ifstream file;
file.open(filename.c_str());
char buf[500] = {0};
file.getline(buf, 500);
while(ros::ok())
{
msg.stamp = ros::Time::now();
msg.data2 = count;
msg.word = buf;
ROS_INFO("stamp.sec = %d", msg.stamp.sec);
ROS_INFO("stamp.nsec = %d", msg.stamp.nsec);
ROS_INFO("data2 = %d", msg.data2);
ROS_INFO("word = %s", msg.word.c_str());
msg_pub.publish(msg);
loop_rate.sleep();
++count;
}
return 0;
}
In addition, I can use absolute path in code or param of launch file to load the file when I use roslaunch to launch the config file. But, when I use default path to load the file, where shuold I put the file when I use roslaunch ?
You are able to search for package paths, so maybe using ros::package::getPath('PKG_NAME') for example, and then you concatenate the config/your_configuration.yaml

SPIFFS change content from specific line in File

I have been working with ESP32 and SPIFFS for a while now. My project will involve changing the content from a specific line in the file when the user need to. The file will always be saved the same format so I know which line will be changed.
My current file is stored like this:
Content inside file:
DeviceNmae
test#test.com.br
123456
button to read
uid from databa
internet ssid
internet pass
When the user changes the internet ssid in the Application, My esp32 will be reading the contento from the database and will detect the change. It will store the incoming change and update the line.
For example, I changed the data to "int ssid now", the database will read and change the "internet ssid" to "int ssid now". I would like to update the content from only that line, but I didn't find nothing on that. If I don't find the solution by updating, I will have to delete all the content from the file and create a new one only to change that line.
I append the data like this:
void funcClass::append_data(String funcName, char Text[]) {
file = SPIFFS.open("/esp_name.txt", FILE_APPEND);
while (connection_state == 1 and funcName == ""){
if (connection_state == 1 and funcName == "" and stop_loop == 0){
for (int i = 0; i < strlen(Text); i++){
char c = Text[i];
SerialBT.write(c);
}
SerialBT.write('\n');
}
stop_loop = 1;
if (SerialBT.available()){
while (SerialBT.available()) {
insert_chars = SerialBT.read();
funcName = String(funcName + insert_chars);
}
stop_loop = 0;
}
}
if (file.print(funcName)){
Serial.print("data was added: ");
Serial.println(funcName);
}else{
Serial.println("data was not added");
return;
}
file.close();
}
``
C doesn't support updating parts of a file.
You could either copy the content of your old file into a new one and change the one line before you write it to the new file.
Or maybe you have a look at the settings class if you are using the arduino framework
(or the NVS api if you are using the ESP IDF)

Unable to read content of a file after writing on ESP8266 ESP-01

I try to do some basic read/write file on SPIFFS on ESP8266 ESP-01. After writing a file, if I try to read it, it always returns 0 read bytes. The same code works well on NodeMCU 10 (ESP12E Module).
It is interesting, that I can upload files to ESP8266-01 SPIFFS (I use vMicro in Visual Studio for development and "Publish Server Data Files (spiffs)") and read them OK with my code. It is only if I write a file, the read does not work for that file. I tried different combinations for ESP-01 "Generic ESP8266 Module" (Flash sizes 1M, 512K, SPIFFS sizes 32K, 64K, 128K) with same result.
The sketch code is here (including many debug statements):
#include <FS.h>
const char* fileName = "/abc.txt";
void setup() {
Serial.begin(115200);
delay(5000);
Serial.println("\n\nStarting...");
if (SPIFFS.begin()) {
Serial.println("begin OK");
if (SPIFFS.format()) {
Serial.println("format OK");
auto fW = SPIFFS.open(fileName, "w");
if (fW) {
Serial.println("Open W OK");
auto nWritten = fW.write((byte*)"aaaAAA", 6);
Serial.printf("Written: %d\n", nWritten);
fW.close();
Serial.println("Closed W OK");
}
else {
Serial.println("Failed opening W");
}
delay(5000);
Serial.println("");
auto fR = SPIFFS.open(fileName, "r");
if (fR) {
Serial.println("Open R OK");
auto fSize = fR.size();
Serial.printf("Size: %d\n", fSize);
Serial.printf("Position: %d\n", fR.position());
if (!fR.seek(0)) {
Serial.println("****Seek failed!");
}
Serial.printf("Position: %d\n", fR.position());
Serial.printf("Available: %d\n", fR.available());
char buffer[20] = { 0 };
auto nRead = fR.readBytes(buffer, fSize);
Serial.printf("nRead: %d\n", nRead);
Serial.printf("Data: [%s]\n", buffer);
fR.close();
Serial.println("Closed R OK");
}
else {
Serial.println("Failed opening R");
}
}
}
}
void loop() {
}
Output is here. Note that size is 6 (which is correct) but nRead is 0, and Data in between [] is empty.
Starting...
begin OK
format OK
Open W OK
Written: 6
Closed W OK
Open R OK
Size: 6
Position: 0
Position: 0
Available: 6
nRead: 0
Data: []
Closed R OK
If I comment out SPIFFS.format() statement (because on second load of the sketch the SPIFFS is already formatted and the file is already there) I cannot even open the file for writing.
So, does SPIFFS on ESP-01 has writing problems, and what can I do to be able to write files there successfully?

Cannot open file on Ubuntu

I'm using Ubuntu 14 and trying to create a script to write files, but I'm getting the 5004 error, every time I try to open a file.
datetime currtime;
bool newcandle;
string terminal_data_path = TerminalInfoString( TERMINAL_DATA_PATH );
string filename = terminal_data_path + "\\MQL4\\Files\\" + "data.csv";
int filehandle;
filehandle = FileOpen( filename, FILE_WRITE | FILE_CSV );
if ( filehandle < 0 ){
Print( "Failed to open the file by the absolute path " );
Print( "Error code ", GetLastError() );
}
else {
Print( "file opened with sucess" );
}
How can I solve this problem on Ubuntu?
UPDATE
I tried to change my file to the following:
string terminal_data_path = TerminalInfoString( TERMINAL_DATA_PATH );
string filename = terminal_data_path + "\\tester\\files\\data.csv";
and just for this
string filename = "\\tester\\files\\data.csv";
and for this
string filename = "\\files\\data.csv";
But I'm still getting error, but this time is 5002 not 5004.
MQL4 Permissions By Design Do Not Allow / Restrict FileIO
There are three directories (with subdirectories) where working files can be placed:
/HISTORY/<current broker> - especially for the FileOpenHistory() function;
/EXPERTS/FILES - common case;
/TESTER/FILES - especially for testing ( ref. during Strategy Tester operations ).
Working with files from other directories is prohibited.
Solution
Adapt your MQL4-code so as to meet this fact and respect pre-Build 762 and post-Build 762 differences ( a "new"-MQL4 file-localisations ).
Update
As posted, your MQL4-code ( whether you share it's updated state or not ) shall better handle exceptions. Have met several suprising artefacts with filenames. Some platform specific, causing no harm in wXP, but failing to operate (the same code) on VPS-hosted wServer2008 VM or a LinuxVM-encapsulated Wine/MT4 instance.
Carefully read MQL4-help documentation and create a few post-mortem tools to step further.
5002
ERR_FILE_WRONG_FILENAME
Wrong file name -------> pre-test + "fuse" the corner cases
5003
ERR_FILE_TOO_LONG_FILENAME
Too long file name
5004 <------ a good sign, we are on the safer side here
ERR_FILE_CANNOT_OPEN
Cannot open file
//-------------------------------------------------------------
// MT4_GUI_postMortem
//-------------------------------------------------------------
void MT4_GUI_postMortem( string aFileNAME = "caller forgot to pass aFileNAME"
){
// SYNTAX
// if ( aFileHANDLE == INVALID_HANDLE ) MT4_GUI_postMortem( filename );
//
int aLastErrorNUM = GetLastError();
Comment( "POST-MORTEM >>> [", aFileNAME, "] Threw error ", aLastErrorNUM );
Print( "POST-MORTEM >>> [", aFileNAME, "] Threw error ", aLastErrorNUM );
return;
}

iOS with monotouch upload wav file fails

I'm writing an iOS app with monotouch.
I'm recording a sound file via the AVrecorder. I create a working wav file(which I check).
I want to send it to my server by passing it as a byte array (seems to be the best way)
The problem is that the file passed to the server is corrupted.
This is how I convert wav to byte []. Any ideas what I'm doing wrong?
(I iterate to find the files size once and than copy everything. The reason I do this is that the GetProperty returns a wrong size of file).
string path = GlobalData.UserDefaults.StringForKey(GlobalData.HeykuSoundPathDefault);
AudioFile song = AudioFile.Open(path,AudioFilePermission.Read,AudioFileType.WAVE);
byte [] audioBuffer = new byte[4000000];
int copied = 0;
int offset = 0;
while(true)
{
copied = song.Read(offset,audioBuffer,0,4096,false);
if(copied == -1)
{
break;
}
else
{
offset += copied;
Console.WriteLine(offset);
}
}
audioBuffer = new byte[offset];
copied = song.Read(0,audioBuffer,0,offset,false);
Have you tried simply using streams? Something like this:
using (var fileReader = new StreamReader("filename_here"))
{
using (var webStream = webClient.OpenWrite("http://www.myuploader.com"))
{
fileReader.BaseStream.CopyToAsync(webStream);
}
}
The above method should copy the whole file including the Wave headers. I suspect the AudioFile could be returning you pure PCM audio which will not include headers. This is an example of a header writer function:
private void WriteHeader()
{
this.writer.Seek(0, SeekOrigin.Begin);
// chunk ID
this.writer.Write('R');
this.writer.Write('I');
this.writer.Write('F');
this.writer.Write('F');
this.writer.Write(this.ByteCount);
this.writer.Write('W');
this.writer.Write('A');
this.writer.Write('V');
this.writer.Write('E');
this.writer.Write('f');
this.writer.Write('m');
this.writer.Write('t');
this.writer.Write(' ');
this.writer.Write((int)16);
this.writer.Write((short)1);
this.writer.Write((short)this.ChannelCount);
this.writer.Write((int)this.SampleRate);
this.writer.Write((int)this.SampleRate*2);
this.writer.Write((short)2);
this.writer.Write((short)this.BitsPerSample);
this.writer.Write('d');
this.writer.Write('a');
this.writer.Write('t');
this.writer.Write('a');
this.writer.Write((int)this.byteCount);
}
More info on the file format: https://ccrma.stanford.edu/courses/422/projects/WaveFormat/
As for reading the data from the AudioFile, this would probably work better:
var song = AudioFile.Open(path,AudioFilePermission.Read,AudioFileType.WAVE);
var audioBuffer = new byte[song.Length];
int copied = 0;
while (copied < audioBuffer.Length)
{
copied += song.Read (copied, audioBuffer, copied, 4096, false);
}

Resources