Author Topic: AT commands coming out in DigiX Wifi requests as client  (Read 8782 times)

jamiecockrill

  • Newbie
  • *
  • Posts: 8
AT commands coming out in DigiX Wifi requests as client
« on: July 10, 2014, 01:34:24 pm »
Hello!

I'm having trouble with HTTP requests I'm making from my DigiX to a python-based web-server. I've had trouble with it working for a few hours and then it drops out. I've tried a number of different ways of writing the post requests (manually with client.connect and client.print, then again with the client.post method) but I've not really had much luck improving it.

When I ran it as a CGI script behind nginx, I was getting "400 - Bad Request" errors interspersed with correctly formed requests. I've since set up the same web-server on my windows machine and run a packet-capture with WireShark (on the server side obviously).

It looks like occasionally I get AT commands in the network traffic. I see:

+++AA+E
AT+TCPDIS=off
AT+NETP
AT+NETP=TCP,CLIENT,80,192.168.0.8
AT+TCPDIS=On
AT+TCPLK

It seems likely that I'm leaking connections. I've tried using client.begin(9600, true) and plain old client.begin() and neither eradicate the issue.

Any help gratefully received! I'm trying to create an RF24Network based home sensor network, where the sensor nodes relay readings to a web-server, via the DigiX.

Code: [Select]
/*
This is a mix of my own work and heavily inspired by RF24 examples from maniacbug.
*/
#include <DigiFi.h>
#include <RF24.h>
#include <RF24Network.h>
#include <SPI.h>
#include "printf.h"

//#define DEBUG /* use this to turn on extra Serial debugging statements */

// DIGIFI SETUP
DigiFi wifi;

// ThingSpeak SETUP
//IPAddress server(192,168,0,202);
char server[] = "192.168.0.8";
uint16_t server_port = 80;

//char server[] = "api.thingspeak.com";
String tempAPIKey = "";
String energyAPIKey = "";
// Variable Setup
int failedCounter = 0;

// RF24 SETUP
// nRF24L01(+) radio attached using Getting Started board
RF24 radio(53,52);

// Network uses that radio
RF24Network network(radio);

// Address of our node
const uint16_t this_node = 0;
// decimal precision. Can't go higher than 2 decimal places
// otherwise we could get integer overflow (8-bit, 255 max value).
const uint8_t decimal_precision = 100;

// Structure of our payload
struct payload_t
{
  uint16_t node_id;
  uint8_t d_value;
  uint8_t p_value;
  uint16_t d_power;
};
//value type is:
// 0: temperature
// 1: power
// etc for more

String sensorData[] = {
  "field1=0",
  "field2=0",
};

String powerData[] = {
  "field1=0",
  "field2=0",
};

int led = 13;

void setup(void)
{
  pinMode(led, OUTPUT);
  Serial.begin(9600);

  //DigiX trick - since we are on serial over USB wait for character to be entered in serial terminal before continuing
  //while(!Serial.available()){
  //  Serial.println("Enter any key to begin");
  //  delay(1000);
  //}
  // END DigiX trick
  Serial.println("DigiX/R24Network/rx/ThingSpeak");
  printf_begin();
  Serial.println("printf begin");
  SPI.begin();
  Serial.println("SPI begin");
  radio.setRetries(15,15);
  radio.begin();
  blink(2);
  Serial.println("radio begin");
  delay(2000);
  network.begin(/*channel*/ 90, /*node address*/ this_node);
  Serial.println("Awaiting data");
  blink(3);
  delay(2000);
  // START turn on wifi
  startDigiFi();
  Serial.println("Connected to network");
  // END turn on wifi
  blink(4);
}

void loop(void)
{
 
  // Pump the network regularly
  network.update();
 
  // Is there anything ready for us?
  while ( network.available() )
  {
    // If so, grab it and print it out
    RF24NetworkHeader header;
    payload_t payload;
    network.read(header,&payload,sizeof(payload));
   
    kkprintf("Received temp: ");
    float a = (float) payload.d_value;
    float b = ((float) payload.p_value)/((float) decimal_precision);
    float t = a + b;
    kkprintf("%.2f Celsius | from Node: %u | with ID: %u\r\n",
      t,
      header.from_node,
      header.id);
   
    if (header.from_node < 3) {
      //just switch whichever of the two readings we are supposed to be sending
      //sensorData[header.from_node-1] = "field"+String(header.from_node)+"="+String(t);
   
      String tData = "node_id="+String(header.from_node)+"&type=T&value="+String(t);
     
      #ifdef DEBUG
      Serial.print("Sending: ");
      Serial.println(tData);
      #endif /* DEBUG */
     
      updateThingSpeak(tData, tempAPIKey);
    }
    if (header.from_node == 3) {
      // Send temperature first
      String tData = "node_id="+String(header.from_node)+"&type=T&value="+String(t);

      #ifdef DEBUG
      Serial.print("Sending: ");
      Serial.println(tData);
      #endif // DEBUG
     
      updateThingSpeak(tData, energyAPIKey);
     
      // Send power next
      String tData2 = "node_id="+String(header.from_node)+"&type=P&value="+String((long)payload.d_power,DEC);

      #ifdef DEBUG
      Serial.print("Sending: ");
      Serial.println(tData);
      #endif // DEBUG
     
      updateThingSpeak(tData2, energyAPIKey);
    }
   
    if (failedCounter > 3){ startDigiFi(); }
  }
}

void blink(int times) {
  for (int i=0;i < times;i++) {
    digitalWrite(led, HIGH);   // turn the LED on (HIGH is the voltage level)
    delay(200);               // wait for a 1/2 second
    digitalWrite(led, LOW);    // turn the LED off by making the voltage LOW
    delay(200);               // wait for a 1/2 second
  }
}

void updateThingSpeak(String tsData, String apiKey)
{
  blink(1);
  if (wifi.post(server,"/data",tsData)) //"192.168.0.202"
  {
    Serial.println("POSTed");
    #ifdef DEBUG
    Serial.println(); 
    Serial.println(wifi.body());
    #endif
    failedCounter = 0;
  }
  else
  {
    failedCounter++;
    #ifdef DEBUG
    Serial.println("Sending data to ThingSpeak failed ("+String(failedCounter, DEC)+")");   
    Serial.println();
    #endif
  }
  blink(2);
}

void startDigiFi()
{
  Serial.println("Connecting Arduino to network...");
  Serial.println(); 

  //wifi.stop();
  delay(1000);
 
  // Connect to network amd obtain an IP address using DHCP
  wifi.begin(9600, true);
  //wifi.begin();
  //wifi.setDebug(true);
  while (wifi.ready() != 1)
  {
    Serial.println("Connecting to network...");
    delay(1000);
  }
  delay(1000);
}

PS - there's a certain amount of junk code in here left over from when I was trying to convert the thingspeak example. I'm not using thingspeak anymore, but a custom-written backend in python (web.py). It's pretty trivial, POST submits data, GET retrieves it and highcharts make it look pretty.


gogol

  • Sr. Member
  • ****
  • Posts: 398
Re: AT commands coming out in DigiX Wifi requests as client
« Reply #1 on: July 11, 2014, 12:12:37 am »
PS - there's a certain amount of junk code in here left over from when I was trying to convert the thingspeak example. I'm not using thingspeak anymore, but a custom-written backend in python (web.py). It's pretty trivial, POST submits data, GET retrieves it and highcharts make it look pretty.

It would be helpful for others, if you clean up the junk before others need to dig out, what is junk and what is essential.
That is usually the way, I start debugging things like that: Trying to reduce the code to the bare minimum, which still causes the problem.  In many if not most cases I find the solution on that way.

On the protocol level it would be helpful, having the network dump available. For that it might be helpful filtering just packets with the MAC from the DigiX on the server, that way you make sure, you are posting no other data.

It sounds for me like a timing problem, back in times where I connected with FidoNet instead of internet.   There we had this problem daily.
http://en.wikibooks.org/wiki/Serial_Programming/Modems_and_AT_Commands/Special_Commands_and_Character_Sequences#.2B.2B.2B:_Escape_Sequence

Unfortunately the wifi module uses slightly modified escape sequences, with a second handshake A (after +++), but the manual gives no timing information, in which time the three + needs to be sent, to be recognized as escape, or in which time the following handshake needs to be completed, that the wifi card switches to command mode.

Chapter 4.1.1 of the wifi modules documentation says:
Quote
  • When user input “+++” (No “Enter” key required), the UART port will display feedback information “a”,
    and not display input information”+++” as above UART display.
  • Any other input or wrong step to UART port will cause the module still works as original mode
    (transparent transmission).

As long, as escaping to command mode fails, the module stays in transparent transmission and all data coming serial in are going out over the current network socket.

So I would try to identify the places, where your code needs to escape to command mode (e.g. to open/close sockets) and have a look, if you could do something there with the timing.


jamiecockrill

  • Newbie
  • *
  • Posts: 8
Re: AT commands coming out in DigiX Wifi requests as client
« Reply #2 on: July 11, 2014, 07:49:58 am »
Quote
So I would try to identify the places, where your code needs to escape to command mode (e.g. to open/close sockets) and have a look, if you could do something there with the timing.

So-far the least-worst performing strategy has been the 'post' method in the library itself, so if I need to fix the timing I'll probably need to root around in there. Which is fine, I can have a look. Indeed I've deliberately not been using the AT command wrapper-methods so far, so I hope to get it to the point where the higher-level methods in the library are a little more reliable. If it looks worthwhile to anyone else I'll create a fork on github.

Quote
Unfortunately the wifi module uses slightly modified escape sequences, with a second handshake A (after +++), but the manual gives no timing information, in which time the three + needs to be sent, to be recognized as escape, or in which time the following handshake needs to be completed, that the wifi card switches to command mode.

Ok - I might e-mail the sales e-mail address in appendix B to see if they have any pointers.

jamiecockrill

  • Newbie
  • *
  • Posts: 8
Re: AT commands coming out in DigiX Wifi requests as client
« Reply #3 on: July 11, 2014, 01:02:19 pm »
Right. Results of a bit more digging. Tracing back through the AT commands and comparing to the library, it appears that the commands I'm seeing look pretty much like what you'd expect after calling the 'connect(IPAddress ip, uint16_t port = 80)' method.

This then rang bells with another post on this forum that I have seen: http://digistump.com/board/index.php/topic,1270.0.html (Problems with Wifi).

I've since had a look at the vendor's website and they have a datasheet for the new version of the wifi module, the USR-WIFI232-G2, which has a little extra step at the top of page 37 which describes the timings required.

http://en.usr.cn/download/USR-WIFI232-G2-V1.4_en.pdf

The only thing I can see that might conflict with the way the library currently works is that I don't see the library checking that the return value is actually an 'a' before it continues. I'm going to double-check the timing and perhaps start looking at putting some logic around checking the timing of the returned value and perhaps retrying the '+++' sequence if we get an unexpected return value from the module.

jamiecockrill

  • Newbie
  • *
  • Posts: 8
Re: AT commands coming out in DigiX Wifi requests as client
« Reply #4 on: July 13, 2014, 06:27:07 am »
I've just committed some changes to DigiFi to my own fork.

https://github.com/jdcockrill/DigiFi/commit/5e5909258925f2c1813e00300f8922850b9e059a

It's not changed quite as significantly as the diff might lead you to believe. In startATMode I've broken the initial bit of the handshake sequence out into a separate private method so I can wrap it in a do-while loop to check that the initialisation of the handshake was successful.

In the startATSequence method I've got some validation in to check that the character returned by the Wifi module is actually an 'a' character as the datasheet says. Doing this means that if it doesn't work first time (or we don't get a reply within the total 3 seconds permitted for the entire handshake), I can report failure to the startATMode method and it can just try again.

I've then had a go with the get and post methods (though I only use post in my code). I've changed them so that they return the HTTP status code (or a negative number if there's some other connectivity/non-HTTP problem) and put a disconnect method at the end which sends the "TCPDIS=off" command to the Wifi module.

I'm now running it for a while to see when/if I get issues. It seems I get more 'Checking for link build up' messages, but once it does build the link, it runs well. It's been going longer than my previous efforts without modifying the library.

Health warning on get, I've modified it to keep it equivalent to post, but it's untested so far as I don't use it. I might finish it off by getting it to return the header and body in char-arrays provided as parameters (similar, but headers only for post).

I'll post with my results once it's been running a while.

gogol

  • Sr. Member
  • ****
  • Posts: 398
Re: AT commands coming out in DigiX Wifi requests as client
« Reply #5 on: July 14, 2014, 12:46:45 am »
I would recommend, that you just open a new thread in "DigiX Libraries" and name it somthing with  "DigiFI optimization", as that seems the right place for your work and upcoming discussion.
Good work!


jamiecockrill

  • Newbie
  • *
  • Posts: 8
[SOLVED] AT commands coming out in DigiX Wifi requests as client
« Reply #6 on: July 15, 2014, 12:19:50 pm »
Done, see here: http://digistump.com/board/index.php/topic,1480.0.html

Hopefully it'll be a useful addition!