I am having an issue with missing serial data whilst using this code:

I am having an issue with missing serial data whilst using this code:

void loop()
{
  while (Serial.available()) {
    buffChar = Serial.read();
    Serial.print(int(buffChar));
    Serial.print(",");
    //if it's not a newline then add to buffer
    if (buffChar != byte(255)) {
      buff[buffPos%sizeof(buff)] = buffChar;
      buffPos++;
    } 
    else {
      //detect newline to end update
      //clear leds
      memset(leds, 0, sizeof(leds));
      //overwrite leds with buffer
      memcpy(leds, buff, sizeof(leds));
      //clear buffer and reset position
      memset(buff, 0, sizeof(buff));
      Serial.println(buffPos);
      buffPos = 0;
      FastLED.show();
    }
  }
}

It works fine most of the time but every 20 or 30 “packets” it gets confused and “ends” the packet early.

I’m using this to send the serial:

import serial, time

ser = serial.Serial('/dev/tty.usbmodem000001', 115200)

for x in range(0,24):
	

	for b in range(0, 64):

		packet = bytearray()

		for led in range(0, 4):
			packet.append(b)
			packet.append(b)
			packet.append(b)

			packet.append(64)
			packet.append(32)
			packet.append(32)

		packet.append(255)

		for p in packet: print(p, ",", end="")
		print("packet length: ", len(packet))

		ser.write(packet)
		time.sleep(0.05)

ser.close()

What leds are you using? The WS2811 chipsets disable interrupts while writing out frame data - which means that if serial data comes in while the interrupts are disabled, some data might get lost.

I would suggest using some kind of marker for the beginning/end of your frame, this way, when you start reading a packet, you discard everything that isn’t your header, and once you see the header pattern, then you start reading packet data.

I’m using a Neopixel stick — so WS2812

The really odd thing is that it always gets the packet size wrong at multiples — so 24 is the right size - but it goes to 36 or 48 — so it’s not random.

Here’s a copy of my debugging serial output to show you what I mean:

62,62,62,64,32,32,62,62,62,64,32,32,62,62,62,64,32,32,62,62,62,64,32,32,255,24
63,63,63,64,32,32,63,63,63,64,32,32,63,63,63,64,32,32,63,63,63,64,32,32,255,24
0,0,0,64,32,32,0,0,0,64,32,32,0,0,0,64,32,32,0,0,0,64,32,32,255,24
1,1,1,64,32,32,1,1,1,64,32,32,1,1,1,64,32,32,1,1,1,64,32,32,255,24
2,2,2,64,32,32,2,2,2,64,32,32,2,2,2,64,32,32,2,2,2,64,32,32,255,24
3,3,3,64,32,32,3,3,3,64,32,32,3,3,3,64,32,32,3,3,3,64,32,32,4,4,4,64,32,32,4,4,4,64,32,32,4,4,4,64,32,32,4,4,4,64,32,32,255,48
5,5,5,64,32,32,5,5,5,64,32,32,5,5,5,64,32,32,5,5,5,64,32,32,255,24
6,6,6,64,32,32,6,6,6,64,32,32,6,6,6,64,32,32,6,6,6,64,32,32,7,7,7,64,32,32,7,7,7,64,32,32,255,36
8,8,8,64,32,32,8,8,8,64,32,32,8,8,8,64,32,32,8,8,8,64,32,32,255,24
9,9,9,64,32,32,9,9,9,64,32,32,9,9,9,64,32,32,9,9,9,64,32,32,255,24
10,10,10,64,32,32,10,10,10,64,32,32,11,11,11,64,32,32,11,11,11,64,32,32,11,11,11,64,32,32,11,11,11,64,32,32,255,36
12,12,12,64,32,32,12,12,12,64,32,32,13,13,13,64,32,32,13,13,13,64,32,32,13,13,13,64,32,32,13,13,13,64,32,32,255,36
14,14,14,64,32,32,14,14,14,64,32,32,14,14,14,64,32,32,14,14,14,64,32,32,255,24
15,15,15,64,32,32,15,15,15,64,32,32,15,15,15,64,32,32,15,15,15,64,32,32,255,24

The number at the end of the row is the count of bytes — it shouldn’t ever go above 24

For shits and giggles, can you try dropping your serial data rate down a bit? Back to 57600, or even 38400 and see if it keeps happening?

(Nice change - usually serial problems like this are a WS2811 vs. interrupt issue - my suspicion now is that you may be sending data a little bit too quickly and something’s getting dropped - though it’s weird that you’re losing data in blocks of 12)

Are you using the 328p? Try something with the ATMega32u4 like the Leonardo or, better yet, the Teensy. They have built in USB which comes with its own buffers and seems to rely much less, if at all, on interrupts for the CDC Serial implementation. On the Teensy I can even get as much as 1.2MBps (Yes, MB not bit) throughput since Paul’s USB Serial code is WAY more optimized than Arduino’s which seems to cap out around 80KBps on the same chip. Using the WS2812 and FastLED I’ve gotten 70FPS with no dropped frames and that’s with 600 pixels, so 1800 bytes per frame, much larger than the 64 byte serial packet size.

I haven’t released it yet as it’s still a bit in flux, but I actually have a modfied Teensy core that will run on any 32u4 based Arduino board (I run mine on a SparkFun Pro Micro). Let me know if you want to try it out and I’ll get you a copy.

Nevada, you can get it here: https://github.com/adammhaile/ATUSB_Core
It’s a bit rough (use the board labeled “ATMega32u4”… I don’t know if the others work yet) and it does not support a bootloader currently (that’s a LONG story). You must program via an ISP like AVR ISP MkII. Also, you have to install the teensy software for it to work. And the pin numbers are the same as the teensy. Like I said, it’s a little rough :stuck_out_tongue: Just look in https://github.com/adammhaile/ATUSB_Core/blob/master/cores/atusb/core_pins.h and find which number matches which port register bit. It will, however, run on any 32u4 based board.

Hopefully I’ll have time to clean it up soon, but for now it’s only intended to run on my custom boards that don’t have all the pins broken out anyways so I dont’ care what the pin numbers are.

Enjoy :slight_smile:

Found out the reason for losing data in blocks of 12 — that’s how big the data packets are on the RF unit I’m using.

I can change this - so will try it with 25bytes.

I’ll also add in some proper checking so the python code will resend a packet if the receiver doesn’t get it.

@James_Carruthers What RF unit are you using?

Ciseco SRF units

@Zeph_Smith

@Zeph_Smith sorry, was alerting you to my reply — I’m using Ciseco RF units

So the solution was to change the packet size to 25 bytes (8 x 3 bytes + 1 byte for end frame) — this made it easier to manage as I didn’t have to manage split “missing” packets.

The sender also now waits for an acknowledgement from the receiver before sending the next frame.

Last thing to do is change that acknowledge byte in to a checksum — but I can’t find a CRC function for both Arduino and Python at the moment.