Serial Port Data Framing Protocol

Solving the core problem of "device continuous transmission causing packet concatenation" and "unable to identify complete frames" in serial communication

📖 Table of Contents:

• Problem Background
• Framing Solution Comparison
• Plan B: Frame Interval Timeout Framing
• Plan C: Protocol Frame Parser
• B + C Combined Practice
• Solution Selection Guide
• Tuning and Troubleshooting

---

Problem Background

Packet Concatenation Phenomenon

In serial communication, devices often continuously send multiple frames of data. The Windows serial driver stacks all received bytes in the receive buffer, without distinguishing frame boundaries. If using a 50ms polling timer to read:

Timeline:
0ms      Device sends Frame1 (10 bytes)
5ms      Device sends Frame2 (8 bytes)      ← No pause between Frame1
12ms     Device sends Frame3 (12 bytes)
...
50ms     Timer triggers Poll()
         Buffer has 30 bytes → ReadExisting() reads all at once
         Result: 3 frames concatenated, unable to parse

Polling Timer Limitations

SetTimer on Windows has a real precision of only ~15ms. Even setting PollInterval to 10ms cannot achieve true real-time. Shortening the polling interval only mitigates, cannot eliminate packet concatenation.

Solution Approach

To identify the boundary of each frame, explicit "framing rules" are needed. This class provides two complementary solutions:

• Plan B: Use the brief pause after the device finishes sending one frame as a natural framing point
• Plan C: Precisely split by device protocol format (delimiter / start-end markers / length field / fixed length)

---

Framing Solution Comparison

Solution	Real-time	Precision	Need Protocol Knowledge	Applicable Scenario
B. Frame Interval Timeout	Good	Medium	No	Device pauses after sending each frame (most scenarios)
C1. Delimiter	Best	High	Yes	Text protocol with fixed delimiter (e.g., \r\n)
C2. Start-End Markers	Best	High	Yes	Binary protocol with STX/ETX markers
C3. Length Prefix	Best	High	Yes	Protocol with length field (e.g., Modbus-like)
C4. Fixed Length	Best	High	Yes	Protocol with fixed frame length
B + C Combined	Best	Highest	Yes	High reliability scenarios, coarse then fine splitting

---

Plan B: Frame Interval Timeout Framing

Principle

After the device finishes sending each frame, there is necessarily a brief pause (even a few milliseconds) before sending the next frame. This pause is the natural frame boundary:

Frame1(10B) |pause| Frame2(8B) |pause| Frame3(12B)
            ↑                  ↑
        Framing point      Framing point

cSerialPort internally maintains a frame buffer; continuously received data is treated as the same frame; when no new data arrives for more than FrameInterval milliseconds, one frame is considered complete, and the FrameReceived event is triggered.

Enabling Framing

Simply set the FrameInterval property to a non-zero value to enable:

Private WithEvents m_Port As cSerialPort

Private Sub Form_Load()
    Set m_Port = New cSerialPort
    m_Port.PortName = "COM3"
    m_Port.Config.BaudRate = br9600

    If m_Port.OpenPort() Then
        ' Enable framing: 30ms no new data = one frame ends
        m_Port.FrameInterval = 30

        ' Polling interval recommended <= FrameInterval / 2, for timeliness
        m_Port.StartMonitoring 20
    End If
End Sub

' Frame received event - triggered once for each complete frame received
Private Sub m_Port_FrameReceived(FrameData() As Byte)
    Dim s As String
    s = StrConv(FrameData, vbUnicode)
    Debug.Print "Received frame (" & UBound(FrameData) + 1 & " bytes): " & s
End Sub

Key Properties

Property	Type	Default	Description
FrameInterval	Long	0	Frame interval timeout (ms), 0=disable framing (keep original DataReceived mode)
MaxFrameSize	Long	65536	Maximum bytes per frame, forced trigger if exceeded, preventing memory overflow from abnormal data
FrameBufferLength	Long	-	Read-only, number of bytes currently in frame buffer (untriggered data received)

FrameInterval Value Recommendations

Device Type	Recommended Value	Description
Text devices (GPS, temperature/humidity, etc.)	20-50ms	Obvious inter-frame pauses
Industrial instruments	10-30ms	Fast response, short pauses
Modbus RTU	-	Not recommended for Plan B, should use Plan C3 (see below)
Slow devices (electronic scales, etc.)	50-100ms	Device sends slowly by itself

Empirical rule: First observe the actual pause between frames when the device continuously sends data (use a serial debug assistant with timestamps), and set FrameInterval to about half of the pause time for the most stable result.

Notes

1. Relationship between polling interval and frame interval: PollInterval should be ≤ FrameInterval / 2, otherwise the frame timeout detection timing may be missed.
2. Does not break original mode: When FrameInterval = 0, it still follows the original DataReceived path, backward compatible.
3. Last frame not lost: StopMonitoring and ClosePort automatically flush remaining untriggered frames in the buffer.
4. Large frame protection: Single frame exceeding MaxFrameSize is forced to trigger, avoiding memory exhaustion from abnormal data.

---

Plan C: Protocol Frame Parser

cSerialFrameParser is an independent frame parsing class that supports 4 protocol framing modes. It does not directly read the serial port, but receives byte streams passed via AppendData, splitting complete frames according to protocol rules.

Reference Method

' Method 1: Source code reference
' Add src/SerialPort/cSerialFrameParser.cls to the project

' Method 2: DLL reference
Dim parser As New VBMANLIB.cSerialFrameParser

C1. Delimiter Mode

Suitable for protocols with fixed delimiters (most common for text protocols):

[DATA1][DELIM][DATA2][DELIM][DATA3][DELIM]...

Dim parser As New cSerialFrameParser

' Frame by CRLF, returned frames do not include delimiter
parser.SetDelimiterMode vbCrLf, False

' Can also use other delimiters
parser.SetDelimiterMode Chr(10), False        ' Only LF
parser.SetDelimiterMode "|", False            ' Custom character

' Inject data
parser.AppendData someBytes

' Loop to extract all complete frames
Do While parser.HasFrame()
    Dim frame() As Byte
    frame = parser.GetFrame()
    Debug.Print StrConv(frame, vbUnicode)
Loop

Parameter description:

Parameter	Description
DelimiterStr	Delimiter string (can be multi-byte, e.g., vbCrLf)
IncludeDelim	True=returned frames include delimiter; False=not included (default)

C2. Start-End Marker Mode

Suitable for binary protocols with STX/ETX and other delimiters:

[STX][DATA1][ETX][STX][DATA2][ETX]...

Dim parser As New cSerialFrameParser

' STX(0x02) ... ETX(0x03), returned frames include markers
parser.SetStartEndMode Chr$(&H2), Chr$(&H3), True

' Also supports multi-byte markers
parser.SetStartEndMode "<<<", ">>>", True

Parameter	Description
StartStr	Start marker string
EndStr	End marker string
IncludeMarks	True=returned frames include markers; False=only return middle data

Fault tolerance: When the buffer start is not StartMark, the parser automatically skips invalid bytes and starts parsing from the next StartMark. This correctly handles the "half-frame residue" problem (unfinished frames from the previous reception).

C3. Length Prefix Mode

Suitable for binary protocols with length fields (e.g., Modbus, custom header protocols):

┌─────────┬───────────┬──────────┬─────────┐
│ Header  │ Length(n) │ Payload  │ Trailer │
│ (fixed) │ (1/2/4B)  │ (n bytes)│ (fixed) │
└─────────┴───────────┴──────────┴─────────┘

Configuration Parameters

Parameter	Description
Offset	Byte offset of the length field in the frame (starting from 0)
Size	Length field size: 1/2/4 bytes
BaseSize	Frame base size (bytes outside the length value range: Header + Length field + Trailer)
BigEndian	True=big-endian; False=little-endian
Meaning	Meaning of the length value (see table below)

LengthMeaning Enumeration

Value	Meaning	Frame Total Length Calculation
lmPayload (0)	Length value = data payload length	FrameBaseSize + length value
lmPayloadAndTrailer (1)	Length value = payload + trailer	FrameBaseSize + length value
lmTotalFrame (2)	Length value = entire frame length	length value

Common Protocol Configuration Examples

Example 1: Simple Length Prefix

[LEN(1)][DATA(n)]

Length value = n (data length), no Header/Trailer.

' Offset=0, Size=1, BaseSize=1(LEN itself), little-endian, lmPayload
parser.SetLengthPrefixMode 0, 1, 1, False, lmPayload

Example 2: With STX and CRC

[STX(1)][LEN(2,BE)][DATA(n)][CRC(1)]

Length value = n, BaseSize = 1(STX) + 2(LEN) + 1(CRC) = 4.

parser.SetLengthPrefixMode 1, 2, 4, True, lmPayload
' Offset=1(skip STX), Size=2 bytes, BaseSize=4, big-endian

Example 3: Length Value Represents Entire Frame

[HEADER(2)][LEN(2,LE)][DATA(n)]

Length value = 2 + 2 + n (entire frame length).

parser.SetLengthPrefixMode 2, 2, 0, False, lmTotalFrame
' Offset=2(skip HEADER), Size=2 bytes, BaseSize=0(not used), little-endian

Example 4: Length Value Includes Trailer

[LEN(1)][DATA(n)][CRC(1)]

Length value = n + 1 (includes CRC), BaseSize = 1 (only LEN field).

parser.SetLengthPrefixMode 0, 1, 1, False, lmPayloadAndTrailer

Modbus RTU Special Note

Modbus RTU has no length field; frame boundaries are identified by a 3.5 character time silent interval (~3.5ms at 9600bps). In this case:

• Recommended to use Plan B (FrameInterval = 4 approx.) for "Modbus-style framing"
• Or directly use VBManLib's cModbusMaster / cModbusSlave classes, which have built-in RTU framing

C4. Fixed Length Mode

Suitable for protocols with fixed frame length (e.g., some sensors output 16 bytes per frame):

parser.SetFixedSizeMode 16   ' Cut one frame every 16 bytes

Do While parser.HasFrame()
    Dim frame() As Byte
    frame = parser.GetFrame()
    ' Process frame...
Loop

Common API

Regardless of the mode, the external interface is unified:

Method/Property	Description
AppendData(Data() As Byte)	Append byte stream to internal buffer
HasFrame() As Boolean	Check if buffer has complete frames
GetFrame() As Byte()	Extract one frame and remove from buffer
GetAllFrames() As Collection	Extract all available frames (each element is a Byte array)
Clear()	Clear internal buffer
BufferLength	Read-only, remaining unparsed bytes in buffer

' Extract all frames at once
Dim frames As Collection
Set frames = parser.GetAllFrames()

Dim v As Variant, frame() As Byte
For Each v In frames
    frame = v
    Debug.Print StrConv(frame, vbUnicode)
Next

---

B + C Combined Practice

Highest reliability scenario: first use Plan B for coarse splitting (time-based), then use Plan C for fine splitting (protocol-based). This way, even if the device has no pause between frames, correct framing can still be achieved.

Typical Scenario

Device protocol: [STX][LEN][DATA][ETX], but the device may continuously send multiple frames without pauses.

Complete Code

Private WithEvents m_Port As cSerialPort
Private m_Parser As cSerialFrameParser

Private Sub Form_Load()
    Set m_Port = New cSerialPort
    Set m_Parser = New cSerialFrameParser

    ' Configure protocol framing (Plan C2: start-end marker mode)
    m_Parser.SetStartEndMode Chr$(&H2), Chr$(&H3), True

    ' Configure serial port
    m_Port.PortName = "COM3"
    m_Port.Config.BaudRate = br115200

    If m_Port.OpenPort() Then
        ' Enable time framing (Plan B)
        m_Port.FrameInterval = 50

        ' Start monitoring
        m_Port.StartMonitoring 20
    End If
End Sub

' Plan B triggers: each "time frame" arrives
Private Sub m_Port_FrameReceived(FrameData() As Byte)
    ' Feed time frame to protocol parser
    m_Parser.AppendData FrameData

    ' Plan C fine splitting: extract multiple protocol frames from one time frame
    Do While m_Parser.HasFrame()
        Dim frame() As Byte
        frame = m_Parser.GetFrame()
        Debug.Print "Complete protocol frame: " & BytesToHex(frame)
    Loop
End Sub

Private Sub Form_Unload(Cancel As Integer)
    If Not m_Port Is Nothing Then
        If m_Port.IsOpen Then m_Port.ClosePort
        Set m_Port = Nothing
    End If
End Sub

' Helper: byte array to hex string
Private Function BytesToHex(buf() As Byte) As String
    Dim i As Long, s As String
    For i = 0 To UBound(buf)
        s = s & Right$("0" & Hex$(buf(i)), 2) & " "
    Next
    BytesToHex = Trim$(s)
End Function

Data Flow Diagram

Device sends:  [STX][LEN=3][A][B][C][ETX][STX][LEN=2][X][Y][ETX]
                              (two frames without pause)

PlanB (FrameInterval=50ms):
  └─ Triggers FrameReceived(12 bytes)

PlanC (SetStartEndMode):
  └─ Extracts two frames:
     1. [STX][LEN=3][A][B][C][ETX]
     2. [STX][LEN=2][X][Y][ETX]

---

Solution Selection Guide

Do you know the device protocol format?
├── No → Use Plan B (FrameInterval = 30~50ms)
│
└── Yes → What features does the protocol have?
          ├── Has delimiter (\r\n, etc.)        → Plan C1
          ├── Has STX/ETX start-end markers     → Plan C2
          ├── Has length field                  → Plan C3
          ├── Fixed length                      → Plan C4
          └── None, but pauses between frames   → Plan B
                                                    │
                                                    └── High reliability requirement?
                                                        ├── Yes → B + C Combined
                                                        └── No → Only B

Quick Reference for Each Solution's Applicable Scenarios

Device Type	Recommended Solution	Configuration Example
GPS module (NMEA 0183)	C1	SetDelimiterMode vbCrLf, False
Temperature/humidity sensor (text output)	C1 or B	Delimiter "\r\n" or FrameInterval=50
RFID reader (STX/ETX protocol)	C2	SetStartEndMode Chr(2), Chr(3), True
Modbus RTU	B or dedicated class	FrameInterval=4, or use cModbusMaster
Custom binary protocol (with length)	C3	See length prefix examples
Electronic scale (fixed length output)	C4	SetFixedSizeMode 16
Industrial instrument (irregular reporting)	B	FrameInterval=30

---

Tuning and Troubleshooting

Common Problems

1. Frame Split in Two Parts

Symptom: Data that should be one frame is split into two FrameReceived triggers.

Cause: FrameInterval is too small; a brief pause during device transmission of one frame is misjudged as frame end.

Solution: Increase FrameInterval. Use a serial debug assistant to first observe the maximum byte interval during device single-frame transmission.

2. Multiple Frames Concatenated into One

Symptom: Data that should be two frames is merged into one FrameReceived trigger.

Cause: FrameInterval is too large; the pause after the device finishes one frame is less than FrameInterval.

Solution: Decrease FrameInterval, or switch to Plan C for precise protocol-based framing.

3. Last Frame Lost

Symptom: After the device finishes sending the last frame and stops, no FrameReceived is triggered.

Cause: After the last frame, there is no subsequent data to "push" the timeout detection.

Solution: This is a fixed issue. StopMonitoring and ClosePort automatically flush the remaining buffer. If you manually manage the lifecycle, ensure calling m_Port.FlushFrames before stopping.

4. Plan C Cannot Parse Frames

Troubleshooting Steps:

' 1. Print raw data for confirmation
Debug.Print "Raw data: " & BytesToHex(rawBytes)

' 2. Check if BufferLength is growing
Debug.Print "Buffer length: " & parser.BufferLength

' 3. Check configuration correctness
Debug.Print "Mode: " & parser.Mode       ' Should be 1/2/3/4
Debug.Print "Buffer: " & parser.BufferLength

' 4. Does protocol match configuration?
'    - Delimiter mode: Check if delimiter bytes are correct
'    - Start-end markers: Confirm STX/ETX byte values
'    - Length prefix: Use debug assistant to view length field value, manually calculate total frame length to verify match

Debug Helper Functions

' Byte array to hex string (for debugging)
Private Function BytesToHex(buf() As Byte) As String
    Dim i As Long, s As String
    On Error Resume Next
    For i = LBound(buf) To UBound(buf)
        s = s & Right$("0" & Hex$(buf(i)), 2) & " "
    Next
    BytesToHex = Trim$(s)
End Function

' Byte array to visible ASCII (invisible characters shown as .)
Private Function BytesToAscii(buf() As Byte) As String
    Dim i As Long, s As String, b As Byte
    On Error Resume Next
    For i = LBound(buf) To UBound(buf)
        b = buf(i)
        If b >= 32 And b < 127 Then
            s = s & Chr$(b)
        Else
            s = s & "."
        End If
    Next
    BytesToAscii = s
End Function

Performance Notes

1. GetFrame() does not copy large blocks of memory: Uses CopyMemory for efficient transfer, MB-level single frames are no problem.
2. Buffer auto-expansion: AppendData auto-expands as needed, no manual management required.
3. Avoid time-consuming operations in events: FrameReceived event callbacks should process as quickly as possible; avoid heavy computation or UI refresh, to prevent affecting next frame reception.

---

Complete API Quick Reference

cSerialPort Framing Related

Member	Type	Description
FrameInterval	Property	Frame interval timeout (ms), 0=disable framing
MaxFrameSize	Property	Maximum bytes per frame (default 65536)
FrameBufferLength	Property	Read-only, current frame buffer byte count
FrameReceived	Event	Triggered when complete frame arrives (parameter: FrameData() As Byte)
FlushFrames	Method	Manually trigger remaining frames in buffer

cSerialFrameParser Complete API

Member	Type	Description
Mode	Property	Framing mode (fmNone/fmDelimiter/fmStartEnd/fmLengthPrefix/fmFixedSize)
BufferLength	Property	Read-only, remaining bytes in buffer
AppendData	Method	Append byte stream
HasFrame	Method	Check if complete frame exists
GetFrame	Method	Extract one frame
GetAllFrames	Method	Extract all frames
Clear	Method	Clear buffer
SetDelimiterMode	Method	Configure delimiter mode
SetStartEndMode	Method	Configure start-end marker mode
SetLengthPrefixMode	Method	Configure length prefix mode
SetFixedSizeMode	Method	Configure fixed length mode

---

Last Updated: 2026-07-05