Author Topic: DigiUSB and Java  (Read 3180 times)

agib

  • Newbie
  • *
  • Posts: 5
DigiUSB and Java
« on: May 04, 2013, 02:59:42 pm »
Hi Everyone!


I've been looking for a way to communicate with the DigiSpark from Java. Java being preferred as it is much easier to deploy than the Python/Ruby based alternatives.


In short, I have been able to successfully send data between the DigiSpark (running the Echo example) and a Java application. However this did require some minor modifications to the DigiUSB library.


I struggled to find a suitable Java USB library, I found many long dead, or poorly documented options. I settled on javahidapi (javahidapi).


javahidapi allows feature reports to be sent/received using sendFeatureReport(byte[]) and getFeatureReport(byte[]). It requires that the first byte in the given array corresponds to the ReportID. This causes problems with the unmodified version of DigiUSB, as it 'misuses' the ReportID value to transfer data. javahidapi would not permit sending anything other than '0' for this value, as the device doesn't provide any ReportIDs and defaults to '0'.


To resolve this I implemented the optional functions usbFunctionWrite and usbFunctionRead in DigiUSB.cpp. Implementing these functions requires USB_CFG_IMPLEMENT_FN_WRITE and USB_CFG_IMPLEMENT_FN_READ to be changed to '1' in usbconfig.h. The changed parts are below:


Code: [Select]
uchar bytesRemaining;


usbMsgLen_t usbFunctionSetup(uchar data[8])
{
  usbRequest_t    *rq = (usbRequest_t*)((void *)data);


    if((rq->bmRequestType & USBRQ_TYPE_MASK) == USBRQ_TYPE_CLASS){    /* HID class request */
        if(rq->bRequest == USBRQ_HID_GET_REPORT){  /* wValue: ReportType (highbyte), ReportID (lowbyte) */
            /* since we have only one report type, we can ignore the report-ID */

return USB_NO_MSG;//we want to use usbFunctionRead()

        }else if(rq->bRequest == USBRQ_HID_SET_REPORT){
            /* since we have only one report type, we can ignore the report-ID */


  // TODO: Check race issues?
bytesRemaining = rq->wLength.bytes[0];//limited to 254 anyway
return USB_NO_MSG;//we want to use usbFunctionWrite()
        }
    }else{
        /* ignore vendor type requests, we don't use any */
    }
    return 0;
}


uchar usbFunctionWrite(uchar *data, uchar len) {
for (uchar i=0; i<len; i++) {
store_char(data[i], &rx_buffer);
}
bytesRemaining -= len;
return bytesRemaining==0;//1 if complete
}


uchar usbFunctionRead(uchar *data, uchar len) {
uchar numBytes = tx_available();
if (len < numBytes) {
numBytes = len;
}
for (uchar i=0; i<numBytes; i++) {
data[i] = tx_read();
}
return numBytes;
}


From the java side, my simple test class consists of the following:

Code: [Select]
import java.io.IOException;
import java.io.InputStreamReader;

import com.codeminders.hidapi.ClassPathLibraryLoader;
import com.codeminders.hidapi.HIDDevice;
import com.codeminders.hidapi.HIDDeviceInfo;
import com.codeminders.hidapi.HIDManager;


public class SimpleUsbTest {

private static final int DIGISPARK_PRODUCT_ID = 0x05DF;
private static final int DIGISPARK_VENDOR_ID = 0x16C0;

public static void main(String[] args) throws IOException, InterruptedException {
ClassPathLibraryLoader.loadNativeHIDLibrary();

for (HIDDeviceInfo info : HIDManager.getInstance().listDevices()) {
System.out.println("Product     : " + info.getProduct_string());//XXX
System.out.println("Manufacturer: " + info.getManufacturer_string());//XXX
System.out.println("Vendor ID   : " + Integer.toHexString(info.getVendor_id()));//XXX
System.out.println("Product ID  : " + Integer.toHexString(info.getProduct_id()));//XXX
}

final HIDDevice digispark = HIDManager.getInstance().openById(DIGISPARK_VENDOR_ID, DIGISPARK_PRODUCT_ID, null);

new Thread(new Runnable() {
public void run() {
byte[] data = new byte[2];
data[0] = 0;//First byte is ReportID

while (true) {
try {
int read = digispark.getFeatureReport(data);
if (read>0) {
System.out.print((char)data[1]);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
}).start();

InputStreamReader in = new InputStreamReader(System.in);

byte[] data = new byte[2];
data[0] = 0;//First byte is ReportID

while (true) {
byte c = (byte)in.read();
data[1] = c;
digispark.sendFeatureReport(data);
}
}
}

The first part simply enumerates the connected devices using listDevices(). For the DigiSpark, it produces the expected:

Code: [Select]
Product     : DigiUSB
Manufacturer: digistump.com
Vendor ID   : 16c0
Product ID  : 5df

The following part opens a device, using the known Vendor/Product IDs. A thread is created which continually reads from the DigiSpark, printing any data it receives.

The final part reads from stdin and sends that data to the DigiSpark.

The samples above currently only send a single byte at a time. To increase this all that is needed on the DigiSpark side is setting the REPORT_COUNT value in the USB report descriptor. The java side just needs the buffers increased correspondingly.

I hope that helps anybody in my situation, the lack of available examples is very frustrating. I'm happy to try and answer any questions. I am working on more fully featured code which makes working with the DigiSpark from within java easier, but I will go into that in another post.

- Andrew

agib

  • Newbie
  • *
  • Posts: 5
Re: DigiUSB and Java
« Reply #1 on: May 04, 2013, 03:04:38 pm »
Forgot to mention, javahidapi supposedly works on Mac, Linux and Windows systems. So a possible benefit of this communication with Java is the ability to write something into the Arduino IDE which replaces the current Serial Monitor functionality, and negates the need for any additional applications.


I would have taken a closer look at this, however my build of the IDE is very old, and I'm having issues getting the new one to build correctly  >:(

Bluebie

  • Sr. Member
  • ****
  • Posts: 487
Re: DigiUSB and Java
« Reply #2 on: May 04, 2013, 06:46:29 pm »
Do you think this could be wrapped up in to a Processing library, with a similar API to the built in Serial support? That would be amazing!!


Your points about misuse of report id seem valid. I think it'd be a good idea to change this behaviour for everyone and change the USB device version number to 2.00. I think there are currently only python, c, and ruby implementations of DigiUSB aside from this one, and I can at least update the ruby implementation and maybe the C ones that get distributed with the digispark IDE these days.


I'm especially interested in seeing this report id problem resolved as it would make it possible to combine mouse, keyboard, joystick, and digiusb in the future in to one modular library, so you could use any combination of them together at the same time. DigiUSB would be a nice way to debug joystick and mouse code in particular.


Cool beans! <3

agib

  • Newbie
  • *
  • Posts: 5
Re: DigiUSB and Java
« Reply #3 on: May 05, 2013, 04:06:35 am »
Certainly. I was intending to write a Java library which provides the basic functionality of discovering devices, or connecting to known devices, and providing implementations of InputStream and OutputStream for communication. There's no reason this couldn't be wrapped in something which provided a Processing-like interface.


I've been looking at the Report IDs because I would like to send more than a single byte at a time. However, when the report count is larger, javahidapi needs exactly that number of bytes to be sent. I'm not sure if this is how it is supposed to work, or just how javahidapi does things. When the DigiSpark is transmitting, it doesn't seem to matter how many bytes it sends.


So to get around this I have a descriptor which defines 2 report types. One with a count of 16 bytes, and the other with a count of 1. This way when I can send data in 16 byte blocks, reverting to a single byte at a time when the data size is smaller. My modifications to DigiUSB currently ignore the Report ID and do the same thing regardless, I'm sure the behaviour could be added to perform the tasks necessary for Mouse/Keyboard/Joystick etc.


Unfortunately I haven't got a great understanding of how the HID interface works, so I can't comment further.

dougal

  • Full Member
  • ***
  • Posts: 201
Re: DigiUSB and Java
« Reply #4 on: May 06, 2013, 09:16:31 am »
Oooh, I need to try your changes out. This might be the key to why I've had problems getting USB comms working under nodejs with the node-hid library (which, in turn, uses the hidapi C library). I've also tried going lower level with node-usb, and was able to read bytes, but not write, for some reason. It may be that this non-conformance is the root of my problems.




Bluebie

  • Sr. Member
  • ****
  • Posts: 487
Re: DigiUSB and Java
« Reply #5 on: May 06, 2013, 05:52:42 pm »
I have two lines of thought on faster digiusb transfers:


1) for download from spark to pc, just set the message pointer to the entire buffer and send it to the PC, which can unpack the struct and read the message, while the spark would reset the start and end points to zero. If this method was used exclusively by the host software it would always end up with the message as a simple byte array because start would stay at zero. This does introduce some nasty tight coupling though in exchange for it's compactness
2) set the usb pointer to the output buffer start address, and set the output bytes length to the number of bytes in the ring buffer, or the number of bytes until the end of the buffer, whichever is shorter. This method would be free of tight coupling, but sometimes it would take two USB requests to clear the whole buffer (one to read till the end of the byte array, and the next to read from the start until the end of the buffer)


I'm happy to implement either in the digispark-device side library and the ruby library, but feel indecisive. What do you guys think?


For writing from PC in to spark, my thinking is we should add two requests - one which asks how many bytes are available in the receiving buffer, and another which writes an arbitrary amount of bytes in - this should be able to totally fill the buffer in two requests, and lets the host device ensure it doesn't overflow the buffer, corrupting the messages, which is much easier to do from the PC side as people are less scared of big strings making their programs huge.


I'd like to also change the behaviour of the library so whenever the device->pc buffer becomes full, any writing function would stall and loop on usbPoll until it can write another byte, then stall again, until it's finished writing out. This means user program would freeze until the computer has emptied out enough buffer, but you would never get corruption from writing messages out faster than the host computer reads them. Maybe there would be some kind of timeout so if the computer doesn't read the device at at at least 50hz or something, this behaviour is disabled. What do people think of this proposed change?

dougal

  • Full Member
  • ***
  • Posts: 201
Re: DigiUSB and Java
« Reply #6 on: May 08, 2013, 11:33:38 am »
I have two lines of thought on faster digiusb transfers:


1) for download from spark to pc, just set the message pointer to the entire buffer and send it to the PC, which can unpack the struct and read the message, while the spark would reset the start and end points to zero. If this method was used exclusively by the host software it would always end up with the message as a simple byte array because start would stay at zero. This does introduce some nasty tight coupling though in exchange for it's compactness
2) set the usb pointer to the output buffer start address, and set the output bytes length to the number of bytes in the ring buffer, or the number of bytes until the end of the buffer, whichever is shorter. This method would be free of tight coupling, but sometimes it would take two USB requests to clear the whole buffer (one to read till the end of the byte array, and the next to read from the start until the end of the buffer)


I'm happy to implement either in the digispark-device side library and the ruby library, but feel indecisive. What do you guys think?

[size=78%]YESYESYES! I think if we used "real" feature reports and could grab output from the spark as a byte array, that would solve my problems trying to use the existing nodejs libraries. Would be maybe be able to tweak the USB message buffer size via a define? It could be useful to have a bigger/smaller buffer, depending on application needs.[/size]


Bluebie

  • Sr. Member
  • ****
  • Posts: 487
Re: DigiUSB and Java
« Reply #7 on: May 09, 2013, 04:57:27 pm »
I haven't had any success using #defines to control the compilation of the digiusb library so far, and I'm not sure why - mangler must be doing something weird. We could put the buffers in the heap though, then they could be defined at startup through the begin function call.

agib

  • Newbie
  • *
  • Posts: 5
Re: DigiUSB and Java
« Reply #8 on: May 11, 2013, 03:46:26 pm »
2) set the usb pointer to the output buffer start address, and set the output bytes length to the number of bytes in the ring buffer, or the number of bytes until the end of the buffer, whichever is shorter. This method would be free of tight coupling, but sometimes it would take two USB requests to clear the whole buffer (one to read till the end of the byte array, and the next to read from the start until the end of the buffer)


I much prefer this solution, as it allows a variable number of bytes to be sent. It would be pointless transmitting the entire buffer for only a few bytes actual data. I have briefly looked at this, however I had trouble working out the order of operations. The data buffer set in usbFunctionSetup() needs to persist when it returns, and I was unsure if control could ever be transferred back to the user program, before the buffer had been cleared.


I'd like to also change the behaviour of the library so whenever the device->pc buffer becomes full, any writing function would stall and loop on usbPoll until it can write another byte, then stall again, until it's finished writing out. This means user program would freeze until the computer has emptied out enough buffer, but you would never get corruption from writing messages out faster than the host computer reads them. Maybe there would be some kind of timeout so if the computer doesn't read the device at at at least 50hz or something, this behaviour is disabled. What do people think of this proposed change?


This makes sense too, however I would prefer to have something similar to available() for the TX buffer. I think this way the operation is more transparent to the user. They can perform the check themselves if they are concerned about overrunning the buffer. I remember a while ago the Arduino developers were discussing something similar for HardwareSerial, but I can't remember the outcome.

Bluebie

  • Sr. Member
  • ****
  • Posts: 487
Re: DigiUSB and Java
« Reply #9 on: May 11, 2013, 04:46:39 pm »
why would a developer on an arduino-based platform ever want the platform to default to confusing corruption of data? when would you ever be sending data to a computer and want it to be corrupt? I don't understand that. Arduino is supposed to be a platform for installation artists to add automations and interactivity to their works.

agib

  • Newbie
  • *
  • Posts: 5
Re: DigiUSB and Java
« Reply #10 on: May 11, 2013, 05:00:35 pm »
From my point of view because doing stuff like that behind the scenes isn't always right, and when it's hidden from the user they may not even be aware it's happening.


It might not always be safe to cause the user program to hang, what if a motor is being controlled, or something else that needs regular attention. In these cases you might want to stop the motor, or do something else to prevent damage. In this situation you would need some level of configuration anyway, which adds to the complexity.


This way it would just be more consistent. If you're writing large amounts of data you can perform the check, if you're only writing a few bytes every once in a while then you probably don't. There'd be nothing stopping someone implementing an interface on top of DigiUSB which performed this checking automatically.

Bluebie

  • Sr. Member
  • ****
  • Posts: 487
Re: DigiUSB and Java
« Reply #11 on: May 11, 2013, 05:19:45 pm »
If you're only writing a few bytes, you don't need to worry because the system will have cleared them out anyway. If you're worried there maybe a buffer overrun and need your program to not freeze until it clears, you can check there are enough free bytes before writing to it. You still have all the same freedoms and performance, the only question is, is the default behaviour to make sure the message gets through, or to make sure the function returns immediately? I think more often than not, people will want the message to go through uncorrupted. With the high speed modifications the buffer will clear as much as 64KB/s - perhaps as much as 128KB/s depending on how the ring buffer is implemented.

dougal

  • Full Member
  • ***
  • Posts: 201
Re: DigiUSB and Java
« Reply #12 on: June 05, 2013, 07:00:32 pm »
Just as an FYI, I applied agib's changes to my DigiUSB library files, and I was then able to use the node-hid module to communicate with my DigiSpark from node.js (finally)!


On the nodejs side, I was able to use the hid.write() method to directly write a buffer (array of bytes) to the spark. But for reading, for some reason the hid.read() method didn't work for me, but I was able to use hid.getFeatureReport(0,254) to fetch the array of characters, then used node's Buffer.toString() method for logging and such.


Now that I have this working, I should be able to do some interesting things with node. I could monitor my web server, and blink an led whenever a page is visited. Or send a tweet based on sensor data.


I'm not sure when I'll have time to do more with it, but I'll get some code posted up eventually.