I have a strange error which I really can't seem to find out. The situation is that I have an arduino board with a temperature and a light sensor: the light sensor is used to see if a certain room is 'open' (light goes out if there's no movement in the room for a certain time). I'm using a Serial port to push data to a server running a Processing script. The arduino board pushes 'OPEN' if the sensed light is above a certain treshhold, 'CLOSED' if it's not. It also pushes the temperature on a newline. It repeats every two seconds.
Monitoring the serial port with minicom that all seems to work fine. Even in the script, the data that I output confirms that everything should work fine. Except that when I try to write data to a 'open.txt' it seems to not to so, while the 'temp.txt' works fine (Tried to tail -f both files, temp.txt gets updated while open.txt just kept being empty.
import processing.serial.*;
Serial mySerial;
PrintWriter openClosedFile;
String openClosedFileName;
String currentOpenClosed;
PrintWriter temperatureFile;
String temperatureFileName;
int currentTemp;
void setup()
{
mySerial = new Serial(this, Serial.list()[0], 9600);
openClosedFileName = "open.txt";
openClosedFile = createWriter(openClosedFileName);
currentOpenClosed = "CLOSED";
temperatureFileName = "temp.txt";
temperatureFile = createWriter(temperatureFileName);
currentTemp = 0;
}
void draw()
{
if (mySerial.available() > 0 )
{
String value = mySerial.readStringUntil('\n');
if ( value != null )
{
String timestamp = nf(day(),2) + "/" + nf(month(), 2) + "/" + year() + " " + nf(hour(),2) + ":" + nf(minute(),2) + ":" + nf(second(),2);
println(timestamp);
value = trim(value);
if (isNumeral(value))
writeTemperature(value);
else
writeOpenClosed(value);
}
}
}
void writeOpenClosed(String val)
{
print("OpenClosed: ");
println(val);
boolean writtenToFile = false;
openClosedFile = createWriter(openClosedFileName);
if (val.equals("OPEN") && !currentOpenClosed.equals("OPEN"))
{
println("val=OPEN and currentOpenClosed!=OPEN");
openClosedFile.print("1");
writtenToFile = true;
}
else if (val.equals("CLOSED") && !currentOpenClosed.equals("CLOSED"))
{
println("val=CLOSED and currentOpenClosed!=CLOSED");
openClosedFile.print("0");
writtenToFile = true;
}
if (writtenToFile)
{
currentOpenClosed = val;
openClosedFile.flush();
openClosedFile.close();
println("Written OpenClosed To File");
}
}
void writeTemperature(String val)
{
print("temperature: ");
println(val);
int intTemp = Integer.parseInt(val);
if (intTemp != currentTemp)
{
currentTemp = intTemp;
temperatureFile = createWriter(temperatureFileName);
temperatureFile.print(val);
temperatureFile.flush();
temperatureFile.close();
println("Written Temperature To File");
}
}
boolean isNumeral(String val)
{
for (int i = 0; i < val.length(); i++)
{
if (val.charAt(i) < 48 || val.charAt(i) > 57)
return false;
}
return true;
}
I expect there to be some syntactical error (I haven't used processing before), but both functions seems to be doing the same...
Some example output:
Listening for transport dt_socket at address: 8212
30/10/2014 12:14:57
OpenClosed: CD
30/10/2014 12:14:57
temperature: 24
Written Temperature To File
30/10/2014 12:14:59
OpenClosed: CLOSED
30/10/2014 12:14:59
temperature: 25
Written Temperature To File
30/10/2014 12:15:01
OpenClosed: CLOSED
30/10/2014 12:15:01
temperature: 24
Written Temperature To File
30/10/2014 12:15:03
OpenClosed: CLOSED
30/10/2014 12:15:03
temperature: 25
Written Temperature To File
30/10/2014 12:15:05
OpenClosed: OPEN
val=OPEN and currentOpenClosed!=OPEN
Written OpenClosed To File
30/10/2014 12:15:05
temperature: 20
Written Temperature To File
30/10/2014 12:15:07
OpenClosed: OPEN
30/10/2014 12:15:07
temperature: 20
30/10/2014 12:15:09
OpenClosed: OPEN
30/10/2014 12:15:09
temperature: 20
^C
Am I not seeing something, or what could be going on here?
Okay, I think I figured it out, it was actually me being dumb.
For eventual future readers: calling
openClosedFile = createWriter(openClosedFileName);
in the writeOpenClosed() function will open the file, and if nothing has to be written it will never get closed. Since the file will never be closed (as there will be a new object made next time the function gets called), it won't release the file to be read.
Related
I have a project where I have data come in via the serial port every 15 minutes. I am using processing to read this data and save it as a CSV.
I would like for a new file to be created every 12 hours. However, when the file switches from AM to PM the entire row gets saved in the PM file (all the previous AM values)
How can I reset the table and start saving to a new file?
saveTable(dataTable, fileName());
dataTable.clearRows();
I tried this but it just clears the CSV file.
String fileName() {
String fileName = "";
String month = "";
String day = "";
int m = month();
int d = day();
if (d < 10) {
day = str(d);
day = "-0" + day;
} else {
day = "-" + str(d);
}
switch(m) {
case 1:
month = "-JAN";
break;
case 2:
month = "-FEB";
break;
case 3:
month = "-MAR";
break;
case 4:
month = "-APR";
break;
case 5:
month = "-MAY";
break;
case 6:
month = "-JUN";
break;
case 7:
month = "-JUL";
break;
case 8:
month = "-AUG";
break;
case 9:
month = "-SEP";
break;
case 10:
month = "-OCT";
break;
case 11:
month = "-NOV";
break;
case 12:
month = "-DEC";
break;
}
if (hour() >= 12) {
hour = "-PM";
} else {
hour = "-AM";
}
fileName = "SensorData_" + str(year()) + month + day + hour + ".csv";
return fileName;
}
Update: Code for collecting and saving data
void serialEvent(Serial myPort) {
if (myPort.available() > 0) {
String serialDataString = myPort.readString();
if (serialDataString != null) {
serialDataString = trim(serialDataString);
float[] sensorData = float(split(serialDataString, ','));
TableRow newRow = dataTable.addRow();
if (sensorData.length == 4) {
temperature = sensorData[0];
humidity = sensorData[1];
moisture = sensorData[2];
int packet = int(sensorData[3]);
if (packet < 10) {
packets = "00" + str(packet);
} else if (packet < 100) {
packets = "0" + str(packet);
}
String time = str(hour()) + ":" + str(minute()) + ":" + str(second());
String date = str(month()) + "/" + str(day());
newRow.setFloat("Temperature", temperature);
newRow.setFloat("Humidity", humidity);
newRow.setFloat("Moisture", moisture);
newRow.setString("Time", time);
newRow.setString("Date", date);
}
saveTable(dataTable, fileName());
}
}
}
In comments you've mentioned
Clearing after a save does not work as expected,
To clarify, what I meant is, if you call clearRows(), previous data will be erased. Saving before clearRows() should save previous data only, saving after clearRows() should only save current data.
I wrote a basic sketch and to me it looks that this works as expected:
void setup() {
// make new table, add 3 cols
Table dataTable = new Table();
dataTable.addColumn();
dataTable.addColumn();
dataTable.addColumn();
// add 1, 2, 3
TableRow newRow = dataTable.addRow();
newRow.setInt(0, 1);
newRow.setInt(1, 2);
newRow.setInt(2, 3);
// save to disk (expecting 1, 2, 3)
saveTable(dataTable,"test1.csv");
// print (expecting 1, 2, 3)
dataTable.print();
// completely clear table
dataTable.clearRows();
// add 4, 5, 6
newRow = dataTable.addRow();
newRow.setInt(0, 4);
newRow.setInt(1, 5);
newRow.setInt(2, 6);
// save again (expecting 4, 5, 6)
saveTable(dataTable,"test2.csv");
// print (expecting, 4, 5, 6)
dataTable.print();
}
(It's also nice that saveTable() appends data (and doesn't overwrite data) in this case.)
This is how I understand how/when data flows in your setup:
Arduino sends data over serial every 15 minutes. You haven't specified if the Arduino has a real time clock (RTC) and the code there uses it to only output data every 15 minutes on the clock (e.g. at :00, :15, :30, :45 past the hour, every hour). The assumption is there is no realtime clock and you're either using delay() or millis() so the actual time data gets sent out is relative to when the Arduino was powered.
When Processing sketch starts, it reads this serial data (meaning any prior data is loest). The assumption is there is no time sync between Arduino and Processing. The first row of data from Arduino comes at Arduino's next 15 minute (not Processing's) after the sketch was started.
The issue you might be experiencing based on your short snippet,
saveTable(dataTable, fileName());
dataTable.clearRows();
if it gets called in serialEvent() is that you'll loose data.
(Confusingly, it doesn't like you're calling clearRows() from serialEvent() ?)
One idea I can think is having some sort of event when the switch from AM/PM to then (first save any accumated data with the previous filename), then clear the the table and update the filename, otherwise (in serial event, save the data with the same filename).
A hacky approach is, once the AM/PM suffixed timestamp is generated to check if this suffix changes and only update filename/clear rows when this change occurs (e.g. manually "debouncing").
Here's a rough sketch to illustrate the idea:
Serial myPort;
float temperature, humidity, moisture;
Table dataTable = new Table();
String packets;
int packet;
boolean isAM,wasAM;
String tableFileName;
public void setup() {
textSize(14);
try{
myPort = new Serial(this, "COM4", 9600);
myPort.bufferUntil('\n');
}catch(Exception e){
println("Error opening Serial port!\nDouble check the Serial port is connected via USB, the port name is correct and the port istn't already open in Serial Monitor");
e.printStackTrace();
}
tableFileName = "SensorData_" + getDateStampString() + ".csv";
}
public void draw() {
background(255);
String sensorText = String.format("Temperature: %.2f Humidity: %.2f Moisture: %.2f", temperature, humidity, moisture);
float textWidth = textWidth(sensorText);
float textX = (width - textWidth) / 2;
fill(255);
rect(textX - 10, 14, textWidth + 20, 21);
fill(0);
text(sensorText, textX, 30);
// get an update date string
String dateStamp = getDateStampString();
// check AM/PM switch and update
isAM = dateStamp.endsWith("AM");
if(!wasAM && isAM){
println("changed PM to AM");
updateTableAMPM(dateStamp);
// update previous state for debouncing
wasAM = true;
}
if(wasAM && !isAM){
println("changed AM to PM");
updateTableAMPM(dateStamp);
wasAM = true;
}
}
public void updateTableAMPM(String dateStamp){
// saves current table (old filename): we're vaing data before the AM/PM switch
saveDataTable();
// clear rows so next 12 cycle starts fresh
dataTable.clearRows();
// update filename (for next save (serialEvent) to use)
tableFileName = "SensorData_" + dateStamp + ".csv";
}
public String getDateStampString(){
return new SimpleDateFormat("yyyy-MMM-dd-aa").format(new Date());
}
public void saveDataTable(){
saveTable(dataTable, tableFileName);
println("saved",tableFileName);
}
public void serialEvent(Serial myPort) {
if (myPort.available() > 0) {
String serialDataString = myPort.readString();
if (serialDataString != null) {
serialDataString = trim(serialDataString);
float[] sensorData = PApplet.parseFloat(split(serialDataString, ','));
TableRow newRow = dataTable.addRow();
if (sensorData.length == 4) {
temperature = sensorData[0];
humidity = sensorData[1];
moisture = sensorData[2];
int packet = PApplet.parseInt(sensorData[3]);
if (packet < 10) {
packets = "00" + str(packet);
} else if (packet < 100) {
packets = "0" + str(packet);
}
String time = str(hour()) + ":" + str(minute()) + ":" + str(second());
String date = str(month()) + "/" + str(day());
newRow.setFloat("Temperature", temperature);
newRow.setFloat("Humidity", humidity);
newRow.setFloat("Moisture", moisture);
newRow.setString("Time", time);
newRow.setString("Date", date);
}
// save data, but don't change the filename
saveDataTable();
}
}
}
Note the above isn't tested (so might contain errors), but hopefully it illustrates the ideas aforementioned.
(One minor note on packets (which I'm unsure where it's used): you can use nf() to easily pad a number with zeros (There are similar functions like nfc(), nfp(), nfs())).
Another option (similar to what I've mentioned in comments) is to use java utilities to call a function after a set time (e.g. the difference in time since the start of the sketch until either noon or midnight, whichever comes first), to then repeat at 12 hour intervals. You can check out TimerTask, or if your familiar with setTimeout in JS you can try this Thread based WIP setTimeout Processing workaround.
I've created a MappedBytes instance to a file that I'm using as shared cache between different Java processes.
I would like to be able to split out additional MappedByte instances (or ByteBuffer or any other instance) from the original that provide direct read/write access to a subset of the underlying file.
I've spent today experimenting with different methods but options like subBytes(), rawCopy() and copyTo() all seem to create local copies of the underlying file, rather than accessing the file directly.
For example:
File tmpFile = new File(System.getProperty("java.io.tmpdir"), "data.dat");
MappedFile mappedFile = MappedFile.mappedFile(tmpfile, 1000, 100, 10, false);
MappedBytes original = MappedBytes.mappedBytes(mappedFile);
original.zeroOut(0, 1000);
original.writeInt(0, 1234);
BytesStore copy = original.bytesStore().subBytes(0, 200);
// Print out the int in the two BytesStores.
// This shows that the copy has the same contents of the original.
System.out.println("Original(0): " + original.readInt(0));
System.out.println("Copy(0): " + copy.readInt(0));
// Now modify the copy and print out the new int in the two BytesStores again.
copy.writeInt(50, 4321);
System.out.println("Original(50): " + original.readInt(50));
System.out.println("Copy(50): " + copy.readInt(50));
Produces the output:
Original(0): 1234
Copy(0): 1234
Original(50): 0
Copy(50): 4321
The copy has been modified but not the original. I would like the original to be modified, can chronicle-bytes do this?
Thanks for your help,
Josh.
This is a self-contained test which I think behaves the way you need.
#Test
public void multiBytes() throws FileNotFoundException {
String tmpfile = OS.TMP + "/data.dat";
MappedFile mappedFile = MappedFile.mappedFile(new File(tmpfile), 64 << 10);
MappedBytes original = MappedBytes.mappedBytes(mappedFile);
original.zeroOut(0, 1000);
original.writeInt(0, 1234);
PointerBytesStore pbs = new PointerBytesStore();
pbs.set(original.addressForRead(50), 100);
// Print out the int in the two BytesStores.
// This shows that the copy has the same contents of the original.
System.out.println("Original(0): " + original.readInt(0));
System.out.println("PBS(0): " + pbs.readInt(0));
// Now modify the copy and print out the new int in the two BytesStores again.
pbs.writeInt(0, 4321);
System.out.println("Original(50): " + original.readInt(50));
System.out.println("PBS(0): " + pbs.readInt(0));
original.writeInt(54, 12345678);
System.out.println("Original(54): " + original.readInt(54));
System.out.println("PBS(4): " + pbs.readInt(4));
}
prints
Original(0): 1234
PBS(0): 0
Original(50): 4321
PBS(0): 4321
Original(54): 12345678
PBS(4): 12345678
I am trying to send a file(.hex file) from my computer to the micro-controller's internal flash. For time being I am using Hercules terminal to send file. My UART responds to the data sent.
My internal flash memory sector is 128Kbytes and my file is about 50Kbytes , so space is not a problem.
While sending .hex file upto a certain point in file the data gets transferred but after a while it stops . I don't understand why.
To slow it down , I have tried my UART baud rate form 115200 to 2400.
Below is the code:
while(1)
{
i = 0;
int c;
char str[256];
printf("\n> ");
do
{
c = fgetc(stdin);
if(c=='\n')
break;
if(c!=-1)
{
str[i++] = c;
delay(10);
}
}while(1);
//str[i]='\0';
//printf("Got..%s\n",str);
int j = 0;
while(j < i-1)
{
uint64_t data;
uint64_t *pData = (uint64_t*)(str + j); //
//data = *((uint64_t*)&str[i]);
//++pData;
data = *pData;
if (HAL_FLASH_Program(TYPEPROGRAM_BYTE, start_address, data) != HAL_OK) {
HAL_FLASH_Lock();
}else
{
//printf("\nSuccess: Writing a byte at (%x) ==> %c ",start_address,*((char*)&data));
}
delay(10);
//data++;
start_address=start_address+1;
j++;
}
}
Below I am attaching my Hercules terminal image :
I have compiled libzmq with openpgm with no changes under windows. Code here is taken from ZeroMQ Guide ("weather publisher" server/client). But if i change "tcp" to "epgm" it doesn't work any more (data is not received, but connection is established).
void test_serv()
{
// Prepare our context and publisher
void *context = zmq_ctx_new();
void *publisher = zmq_socket(context, ZMQ_PUB);
int rc = zmq_bind(publisher, "epgm://127.0.0.1:5556");
assert(rc == 0);
// Initialize random number generator
srandom((unsigned)time(NULL));
while (!stop_server)
{
// Get values that will fool the boss
int zipcode, temperature, relhumidity;
zipcode = randof(1000) + 600;
temperature = randof(215) - 80;
relhumidity = randof(50) + 10;
// Send message to all subscribers
char update[20];
sprintf(update, "%d %d %d", zipcode, temperature, relhumidity);
s_send(publisher, update);
}
LOG("END Server shutdown");
Sleep(500);
zmq_close(publisher);
zmq_ctx_destroy(context);
}
void test_sock()
{
// Socket to talk to server
LOG("Collecting updates from weather server...");
void *context = zmq_ctx_new();
void *subscriber = zmq_socket(context, ZMQ_SUB);
int rc = zmq_connect(subscriber, "epgm://127.0.0.1:5556");
assert(rc == 0);
// Subscribe to zipcode, default is NYC, 10001
char *filter = "1001 ";
rc = zmq_setsockopt(subscriber, ZMQ_SUBSCRIBE,
filter, strlen(filter));
assert(rc == 0);
// Process 100 updates
int update_nbr;
long total_temp = 0;
for (update_nbr = 0; update_nbr < 10; update_nbr++) {
char *string = s_recv(subscriber);
int zipcode, temperature, relhumidity;
sscanf(string, "%d %d %d",
&zipcode, &temperature, &relhumidity);
total_temp += temperature;
LOG(">> " << string);
free(string);
}
LOG("Average temperature for zipcode "<< filter << "was " << (int)(total_temp / update_nbr) << 'F');
zmq_close(subscriber);
zmq_ctx_destroy(context);
}
I run two functions in different threads, with tcp anything works as expected.
I have tried doing "route print 0.0.0.0" with cmd.exe and using interface IP (192.168.137.64) as prefix instead of "eth0" like shown in RFC: epgm://192.168.137.64;127.0.0.1:5556 on connect and/or bind, but this brokes my socket and raises error.
Also "PGM" requires administrator rights and i cannot test it now.
The error IS NOT "protocol not supported" errno is set to B (11) and i don't understand what does it mean (no docs on it).
EPGM is a bit finicky. According to this list post, if you're using EPGM your publisher and subscriber must be on separate hosts. More details here, it looks like this was a deliberate choice by the ZMQ team.
So, try it by spinning up your PUB and SUB on separate machines (changing the network addresses accordingly, of course).
The reason might be that windows does not support loopback capture interface. I tried weather example with protocol changed to epgm on linux and it works fine (well, shows some warnings about loopback, but the messages are transfered correctly)
This is code to run automated blinds. However, it appears that there is some memory leak going on inside of loop as my program will not run. I am using XOBXOB to input data. Look below for where it really breaks.
void loop()
{
Serial.println("loop");
// New XOB request every 4 seconds (if previous response has been received)
if (lastResponseReceived && (abs(millis() - lastRequestTime) > 4*1000)) {
// if the connection has dropped, reconnect
while (!XOB.connected()) XOB.connect();
// Reset timer and response flags. Then, request "XOB" contents
lastResponseReceived = false;
lastRequestTime = millis();
Serial.println("requesting xob");
//enter name of XOB
XOB.requestXOB("XOB");
Serial.println("XOB requested!");
}
// Load response a character at a time when it is available.
// If loadStreamedResponse returns true, a completed response has been received
// Get the "switch" message from the XOB and turn the LED on/off
// NOTE: The message contents are returned with quotes. So, include them
// in the comparison statement.
// Serial.println("LSR: ");
// Serial.print(XOB.loadStreamedResponse());
This is where the loop does not run. It seems that the stream of data from XOBXOB cannot be stored and therefore loadStreamedResponse returns false.
if (!lastResponseReceived && XOB.loadStreamedResponse()) {
Serial.println("requesting reponse");
lastResponseReceived = true;
Serial.println("Get Xob Messages");
//Gets the string from value.
String str_value = XOB.getMessage("value");
Serial.println(str_value);
//Gets the string from switch
String str_switch = XOB.getMessage("switch");
Serial.println(str_switch);
String str_text = XOB.getMessage("text");
Serial.println(str_text);
//If time is not set, get the time from text xob.
if(time_set == false) {
Serial.println("setting time");
// insert code to parse time and set it from the text xob.
time_set = true;
}
// parse the hour from value and convert to minute for wake hour
String string_hour = str_value.substring(1,3);
int wake_hour = string_hour.toInt();
//Serial.println(wake_hour);
// parse minute from value and convert to int for wake minute
String string_minute = str_value.substring(3,5);
int wake_minute = string_minute.toInt();
// Serial.println(wake_minute);
// compare wake_minute and wake_hour with current time
// convert both to minutes for comparison
int wake_time = wake_minute + (60 * wake_hour);
int current_time = 738; //minute() + (hour() * 60);
Serial.println("checking wake condtions");
// rise up blinds slowly if the wake time is + or - 1 minute and if the blinds are closed.
if((wake_time == current_time || wake_time == (current_time - 1) || wake_time == (current_time + 1))
&& blinds_open == false ) {
Serial.println("Natural Wakeup");
rollup_blinds_slow();
blinds_open = true; // blinds should be open now.
}
// Check for opening up blinds condition. Blinds must be closed, to open them
Serial.println("Checking Open Condition");
if (str_switch.equalsIgnoreCase("\"ON\"") && blinds_open == false) {
Serial.println("Blinds Up");
rollup_blinds_fast();
blinds_open = true;
Serial.println(blinds_open);
}
Serial.println("Check closing condition");
// Check for closing blinds condition. Blinds must be closed, to open them
if (str_switch.equalsIgnoreCase("\"OFF\"") && blinds_open == true) {
Serial.println("Blinds down");
rolldown_blinds_fast();
blinds_open = false;
}
}