33. C37.118 synchrophasor ingest¶

This chapter specifies how GridSim reads IEEE C37.118.2 synchrophasor streams: the frame decoder, the CONFIG-1/2 parser, the phasor data concentrator (PDC), the sample-quality rule, the PDC-to-MeasurementSet mapper and its channel-map file, the live TCP / UDP receiver and its quarantine, the linear PMU state estimator and its adapter, and the two verbs, pmu-ingest and pmu-estimate. The code lives in src/GridSim.Core/Synchrophasor/, src/GridSim.Core/Estimation/LinearPmuStateEstimator.cs and PmuMeasurementSetAdapter.cs; the internal HybridMultiRateScheduler is in src/GridSim.Core/Simulation/.

The whole subject is experimental. The path that runs end to end is: gridsim pmu-ingest <capture.bin> (or --live from a device) frames the stream, checks every CRC, parses each CONFIG frame, decodes every multi-PMU DATA frame against the configuration in force for it, refuses the run if any frame is newer than the horizon, drops every sample the quality rule rejects, aligns the rest on their reporting instants with the PDC and prints a summary; with a channel map it writes one gridsim-measurements/1 set per aligned instant through the mapper (and so through the horizon guard again). gridsim pmu-estimate <case> --measurements <set.json> reads such a set and runs the linear PMU estimate. The design record, which also describes parts that are not built, is IEEE C37.118.2 Synchrophasor Ingest + Phasor Data Concentrator; this chapter specifies only what the code does.

Frames are decoded in engineering units (volts, amperes, hertz, hertz per second). A PMU angle is the PMU's own angle, referenced to a UTC-synchronised nominal-frequency cosine, not to any bus of a case; the mapper is where angles are re-referenced to the estimator's slack frame. Chapter 6 (State estimation) specifies the WLS estimator that consumes MeasurementSet rows, and chapter 18 (Real-time distribution and dynamic estimation) places this subsystem among the resident estimators.

Model¶

What is wired¶

Stage	Type	Called by
Frame header and CRC	`C37118CommonHeader`, `C37118Crc`, `C37118Decoder.VerifyCrc`	`pmu-ingest`, the decoder, the config parser
CONFIG-1/2 parse	`C37118ConfigParser.Parse`	`pmu-ingest`
DATA decode, whole stream	`C37118Decoder.DecodeData(frame, C37118ConfigFrame)`	`pmu-ingest`
DATA decode, single PMU	`C37118Decoder.DecodeData(frame, timeBase, PmuConfig)` and the float-only overload	tests only
Sample quality	`C37118SampleQuality.Of(timeQuality, stat)`	`pmu-ingest`
Alignment	`PhasorDataConcentrator`	`pmu-ingest` (wait window zero, one `FlushAll`)
Channel map file	`PmuChannelMapFile.Load`, `ValidateAgainst`	`pmu-ingest --channels`
Mapping to estimator rows	`PmuMeasurementSetMapper.Map`	`pmu-ingest --channels ... --measurements-dir`
Live receive	`C37118LiveReceiver.ReceiveTcp`, `ReceiveUdp`	`pmu-ingest --live`
Live quarantine	`PmuLiveQuarantine`	`pmu-ingest --live ... --live-quarantine`
Polar set to rectangular phasors	`PmuMeasurementSetAdapter.FromSet`	`pmu-estimate`
Linear estimate	`LinearPmuStateEstimator.Estimate`	`pmu-estimate`
Update schedule	`HybridMultiRateScheduler.Schedule` (internal)	tests only

PhasorDataConcentrator, PmuMeasurementSetMapper and LinearPmuStateEstimator carry the marker <b>Experimental</b> (R062, October 2026) in their type comments; HybridMultiRateScheduler is marked EXPERIMENTAL (R095). R062EstimationWiringTests.Stale_docs_are_corrected_and_test_only_classes_are_marked_experimental pins the first three markers by reading the source files.

Common header (14 bytes, big-endian)¶

C37118CommonHeader.Parse reads the start of every frame:

Offset	Bytes	Field	Handling
0	1	SYNC byte 0	Must be `0xAA`; any other value throws `ArgumentException`.
1	1	SYNC byte 1	Bits 6..4 are the frame type, bits 3..0 the version. Bit 7 is ignored; the version is read and not validated.
2	2	FRAMESIZE	Whole frame in bytes, header to CHK inclusive.
4	2	IDCODE	Stream, PDC or PMU id.
6	4	SOC	Seconds since 1970-01-01T00:00:00Z.
10	4	FRACSEC	Top byte: message time-quality flags. Low 24 bits: fraction-of-second count.

A buffer shorter than 14 bytes throws ArgumentException. Frame types (C37118FrameType):

Value	Type	Decoded
0	`Data`	yes
1	`Header`	no
2	`Config1`	yes, same layout as CONFIG-2 for the fields read
3	`Config2`	yes
4	`Command`	no
5	`Config3`	no

Values 6 and 7 are read as undefined enum values and treated as "other" by pmu-ingest. The time-quality byte is stored in C37118CommonHeader.TimeQuality and read by nothing else.

Time¶

The reporting instant is computed by C37118Decoder for every overload with the same expression:

ticks   = SOC * 10^7 + floor(FRACSEC * 10^7 / TIME_BASE)          (integer 100 ns ticks, 64-bit)
TimeUtc = UnixEpoch + ticks

TIME_BASE comes from the CONFIG frame, masked to its low 24 bits (0x00FFFFFF). Both products are exact in a 64-bit integer (SOC is below 2^32 and FRACSEC below 2^24), so the only rounding is the one floor, which acts only when TIME_BASE does not divide FRACSEC * 10^7 (a time finer than one tick); the instant is then the last whole tick at or before the true time. At TIME_BASE = 1,000,000 every FRACSEC count lands on its exact tick (820,000 is .82 s). A TIME_BASE of 0, or one wider than 24 bits, is refused by every decode overload (ArgumentOutOfRangeException naming TIME_BASE), and a DATA frame whose FRACSEC is not below TIME_BASE is Malformed.

CRC-CCITT¶

C37118Crc.Compute is CRC-16/CCITT-FALSE: width 16, polynomial 0x1021, initial value 0xFFFF, no reflection, no final XOR, bit-serial:

crc = 0xFFFF
for each byte b:  crc = crc XOR (b << 8)
                  repeat 8 times:  crc = (crc & 0x8000) ? ((crc << 1) XOR 0x1021) : (crc << 1)     (16-bit)

The check vector "123456789" gives 0x29B1. C37118Decoder.VerifyCrc(frame) reads FRAMESIZE, returns false when the buffer is shorter than 4 bytes, when FRAMESIZE is below 4 or when FRAMESIZE exceeds the buffer, and otherwise compares the last two bytes of the declared frame (CHK) with the CRC of every byte before them. Bytes after the declared frame are ignored.

CONFIG-1/2 frame¶

C37118ConfigParser.Parse reads, after the common header:

Field	Bytes	Kept as
TIME_BASE	4	`C37118ConfigFrame.TimeBase` (low 24 bits)
NUM_PMU	2	number of PMU blocks that follow
per PMU: STN	16	`PmuConfig.StationName` (ASCII, trailing spaces trimmed)
IDCODE	2	`PmuConfig.IdCode`
FORMAT	2	`PmuConfig.Format` (low 4 bits)
PHNMR, ANNMR, DGNMR	2 each	`PhasorCount`, `AnalogCount`, `DigitalCount`
CHNAM	`16 * (PHNMR + ANNMR + 16 * DGNMR)`	skipped
PHUNIT	4 per phasor	`PmuConfig.PhasorUnits`, verbatim
ANUNIT, DIGUNIT	4 per analogue, 4 per digital word	skipped
FNOM	2	bit 0 set gives `Is50Hz`; `NominalHz` is 50 or 60
CFGCNT	2	`ConfigCount`
DATA_RATE	2, signed	`C37118ConfigFrame.DataRate` (frames per second when positive, seconds per frame when negative)

A CONFIG frame sizes and scales every DATA frame that follows, so the parser checks it before trusting its fields. A TIME_BASE of 0 is refused by name. The checks run in this order, and every failure after the header checks throws C37118FrameException, which derives from ArgumentException and names what is wrong and where:

The common header parses (else ArgumentException), and the type is Config1 or Config2 (else ArgumentException: "not a CONFIG frame").
FRAMESIZE is at least 24 bytes: header 14, TIME_BASE 4, NUM_PMU 2, DATA_RATE 2, CHK 2.
FRAMESIZE does not exceed the bytes supplied (a truncated capture or a short read).
The trailing CHK equals the CRC-CCITT of the frame before it. The message carries both check words.
Every later read lies inside the declared frame with CHK excluded. The fixed part of each PMU block, then its variable part (names, units, FNOM, CFGCNT), is size-checked whole before it is read, so a count that runs past the frame never allocates a PHUNIT table. The message names the PMU block (k of n) and the field.
DATA_RATE (2 bytes) fits between the last PMU block and CHK.

Bytes left between DATA_RATE and CHK are accepted: the CRC vouches for them as sent.

FORMAT word¶

The FORMAT value is cast straight into C37118DataFormat after masking with 0x0F. The flag values are the IEEE C37.118.2-2011 bit positions:

Bit	Flag	Value	Set means
0	`PhasorPolar`	1	phasors are magnitude and angle, else real and imaginary
1	`PhasorFloat`	2	phasors are 32-bit IEEE floats, else 16-bit integers
2	`AnalogFloat`	4	analogue values are 32-bit floats, else 16-bit integers
3	`FreqFloat`	8	FREQ and DFREQ are 32-bit floats, else 16-bit integers

None (0) is fixed 16-bit rectangular phasors with integer frequency and analogues. A word such as 0x000E is float rectangular phasors, float analogues and float FREQ/DFREQ.

DATA frame¶

A PMU block is, in order: STAT (2 bytes), the phasors, FREQ and DFREQ, the analogue values, the digital words. Its size, which C37118Decoder uses both to check the frame length and to step to the next block, is:

block bytes = 2 + PHNMR * phasorSize + freqSize + ANNMR * analogSize + 2 * DGNMR
phasorSize  = 8 if PhasorFloat else 4          freqSize   = 8 if FreqFloat else 4     (FREQ and DFREQ together)
analogSize  = 4 if AnalogFloat else 2
frame bytes = 14 + sum of block bytes over the PMUs, in CONFIG order + 2

Every value is decoded into engineering units as follows. scale_i = (PHUNIT_i & 0x00FFFFFF) / 100000.0 (PmuConfig.PhasorScale: one correctly rounded division; 100000 is exact in binary floating point), which is volts or amperes per count.

float phasor, rectangular   real = float32 at +0       imag = float32 at +4
float phasor, polar         mag  = float32 at +0       ang  = float32 at +4 (radians)
integer phasor, rectangular real = int16 at +0 * scale_i       imag = int16 at +2 * scale_i
integer phasor, polar       mag  = uint16 at +0 * scale_i      ang  = int16 at +2 * 1e-4 (radians)
polar to rectangular        Phasor.FromPolar:  real = mag * cos(ang)     imag = mag * sin(ang)

float FREQ / DFREQ          FreqHz = float32            RocofHzPerS = float32
integer FREQ / DFREQ        FreqHz = NominalHz + int16 / 1000.0        RocofHzPerS = int16 / 100.0

An integer FREQ is a deviation from FNOM in millihertz and an integer DFREQ is ROCOF times 100, which is why an integer frame cannot be decoded without the CONFIG frame. Analogue values and digital words are stepped over, not decoded. Phasor.Magnitude is sqrt(Real^2 + Imag^2) and Phasor.AngleRad is atan2(Imag, Real).

The STAT word is returned verbatim in C37118DataFrame.Stat and C37118PmuBlock.Stat; the decoder does not interpret its bits. C37118SampleQuality does, for pmu-ingest: STAT bits 15-14 (data error), 13 (PMU sync), 8-6 (time quality) and 5-4 (unlocked time) decide whether a block is used, together with the frame's FRACSEC time quality (see Sample quality). pmu-ingest also counts how many STAT words are non-zero.

Decode entry points¶

Overload	Reads	Refuses
`DecodeData(frame, timeBase, phasorCount, format)`	one PMU block, float formats only, zero analogue and digital channels	an integer-phasor format (when `phasorCount > 0`) or an integer FREQ format: `ArgumentException`, because integer fields need FNOM and PHUNIT
`DecodeData(frame, timeBase, PmuConfig)`	the first PMU block, any format, with that PMU's analogue and digital sizes	an integer-phasor `PmuConfig` without a complete PHUNIT table: `ArgumentException`
`DecodeData(frame, C37118ConfigFrame)`	one block per PMU of the CONFIG frame, in its order	a CONFIG frame with no PMU, or any integer-phasor PMU without a complete PHUNIT table: `ArgumentException`, before a byte is read

The two single-PMU overloads read only the first block of a multi-PMU frame; a stream with more than one PMU must use the stream overload. Every overload returns a status instead of throwing for frame-level defects (FrameDecodeStatus):

Status	Meaning
`Ok`	CRC verified, frame is a DATA frame and long enough.
`CrcMismatch`	`VerifyCrc` failed. Nothing else in the frame is read.
`NotDataFrame`	The CRC passed and the type is not `Data`.
`Malformed`	The frame is too short for its configuration. The single-PMU overloads require `Length >= need` and `FRAMESIZE >= need`; the stream overload requires `Length >= need` and `FRAMESIZE == need` exactly, so a frame with the wrong number of blocks is refused whole, never half-read.

The order of checks is CRC, then type, then length, so a corrupt frame is never classified by its (untrusted) header fields. C37118StreamDataFrame.StreamIdCode is the frame header's IDCODE, 0 when the CRC failed. Each C37118PmuBlock takes its IDCODE and station name from the CONFIG frame; a block carries no id of its own.

Phasor data concentrator¶

PhasorDataConcentrator(expectedPmuIds, waitWindow) aligns PMU samples (PmuSample: id, TimeUtc, phasors, frequency, ROCOF) onto reporting instants. It is pure transport, deterministic and not synchronised for concurrent use.

Ingest. The bucket key is TimeUtc.Ticks. A later sample for the same PMU and instant replaces the earlier one. A sample whose key is at or before the high-water mark (the last emitted instant) is late: it is discarded, counted in LateSamples and Ingest returns false. Otherwise it returns true.
Flush. Flush(watermark) computes cutoff = watermark - waitWindow and emits, in ascending time order, every bucket whose key is at or before cutoff. Each emitted PdcSuperframe holds TimeUtc, the samples keyed by PMU id, and Dropped: the expected ids with no sample at that instant, in ascending order. Emitted buckets are removed and the last emitted key becomes the high-water mark. FlushAll() is Flush(DateTime.MaxValue).
Expected ids. Duplicates in expectedPmuIds collapse. A sample from an id that is not expected is kept in the superframe and is never listed as dropped.

A superframe is emitted once. A straggler for an emitted instant cannot recreate its bucket, so no second superframe for the same instant, holding only the straggler, ever follows.

Mapping a superframe to estimator rows¶

PmuMeasurementSetMapper.Map(frame, maps, baseMva = 100.0, caseName = "pmu", utcNow = null, angleReference = null) builds a MeasurementSet (src/GridSim.Core/Estimation/MeasurementSet.cs; the row type is in Measurement.cs). A PmuChannelMap lists, per PMU, voltage channels (PmuVoltageChannel: phasor index, bus id, line-to-neutral BaseVolts, VmSigmaPu, VaSigmaDeg) and current channels (PmuCurrentChannel: phasor index, from and to bus, optional circuit, BaseAmps, IMagSigmaPu, IAngSigmaDeg). The row for each channel is:

voltage channel at bus b:   VmPu   = |V| / BaseVolts                        sigma = VmSigmaPu     Bus = b
                            VaDeg  = slackFrameDeg(V)                        sigma = VaSigmaDeg    Bus = b
current channel on a->b:    IMagPu = |I| / BaseAmps                          sigma = IMagSigmaPu   From = a, To = b, Circuit
                            IAngDeg = slackFrameDeg(I)                       sigma = IAngSigmaDeg  From = a, To = b, Circuit

slackFrameDeg(P) = wrap( angle(P) - angle(reference) ) * 180 / pi + AnchorDeg
wrap(d):  d - 2*pi if d > pi;   d + 2*pi if d <= -pi;   else d                  (result in (-pi, pi])

Every row has Source = "pmu". The set carries Case = caseName, BaseMva, TakenUtc = frame.TimeUtc and the note mapped from C37.118 PDC superframe. Rows are emitted in maps order, voltages before currents within a PMU, so the output order does not depend on dictionary enumeration. A PMU in maps that is absent from frame.Samples (dropped at this instant) contributes no rows.

Angle re-referencing. A C37.118 angle is absolute against a UTC-synchronised cosine at nominal frequency, so it rotates with any frequency deviation; the estimator's VaDeg and IAngDeg rows are measured from the slack bus. PmuAngleReference(PmuId, PhasorIndex, AnchorDeg = 0) names one voltage channel whose angle every other angle is measured against. The common UTC-frame offset cancels in the difference; each angle is its own atan2, so the cancellation is not bit for bit (the test holds it to 9 decimal places for a 1.234 rad rotation). AnchorDeg is the slack-frame angle of the bus the reference channel measures: 0 when the channel is at the slack bus. The mapper fails loud:

Condition	Result
Frame time newer than the horizon	`ForwardInferenceException`, before anything else
`angleReference` names a PMU or phasor that is not a mapped voltage channel	`ArgumentException`, before any data is read
An angle row is needed and no `angleReference` was given	`InvalidOperationException` naming the UTC reference
The reference PMU did not report at this instant	`InvalidDataException` ("did not report"), no absolute angle is emitted
The reference PMU's sample has no such phasor	`InvalidDataException`

Magnitude rows need no reference. A superframe that yields no angle rows (every mapped PMU dropped) needs none.

Linear PMU state estimator¶

LinearPmuStateEstimator.Estimate(model, voltages, currents) solves the rectangular state in one non-iterative weighted-least-squares step. With x = [Vre_0..Vre_(n-1), Vim_0..Vim_(n-1)] over the n buses of the model, both phasor kinds are linear in x. A PmuVoltageMeasurement (bus, per-unit phasor, Sigma, CovOffDiag) gives the row pair Vre_k = z_re, Vim_k = z_im. A PmuCurrentMeasurement (from, to, optional circuit, current into the from end, Sigma, CovOffDiag) gives, with the branch stamps Yff, Yft:

Ire = gff*Vre_f - bff*Vim_f + gft*Vre_t - bft*Vim_t
Iim = bff*Vre_f + gff*Vim_f + bft*Vre_t + gft*Vim_t        (Yff = gff + j*bff, Yft = gft + j*bft)

ys  = 1 / (R + jX)      bc = j * B/2      a = TapRatio if > 0 else 1      tap = a * exp(j * PhaseShiftDeg * pi/180)
Yff = (ys + bc) / a^2   Yft = -ys / conj(tap)

These are the pi-model stamps of YBus.Build. The branch is the in-service branch with the given from and to ids and, when Circuit is set, that exact Name, or the ordinal form #k (model branch index k) when no branch is named so; no match throws InvalidDataException, and two matches without a circuit throw InvalidDataException asking for one.

Each phasor contributes a real and an imaginary row with the 2x2 error covariance R = [[sRe^2, c], [c, sIm^2]], sRe = Sigma, sIm = SigmaIm (null: Sigma), c = CovOffDiag. Before any row is built Estimate refuses, naming every problem at once (InvalidDataException): a voltage at a bus the model does not have, a sigma that is not a finite number above 0, a covariance that is not finite or whose matrix is not positive definite (determinant sRe^2 * sIm^2 - c^2 at or below 0), and a phasor value that is not finite. Its inverse enters the normal equations:

R^-1 = [[a, b], [b, d]]   with   det = sRe^2*sIm^2 - c*c;   a = sIm^2/det;   b = -c/det;   d = sRe^2/det

G   = sum over pairs of  H^T R^-1 H            g = sum over pairs of  H^T R^-1 z
x   = G^-1 g                                   (explicit Gauss-Jordan inverse, DseLinAlg.Invert, then MatVec)
J   = sum over pairs of  r^T R^-1 r,   r = z - H x
Vm_i = |Vre_i + j*Vim_i|        VaDeg_i = atan2(Vim_i, Vre_i) * 180/pi

Converged is false when DseLinAlg.Invert finds the gain singular (a pivot at or below NumericTolerances.RelativePivotFloor * max|G_ij|, RelativePivotFloor = 1e-12); the result then has zero Vm and VaDeg and ObjectiveJ = NaN. ObjectiveJ is zero to rounding for a consistent phasor set. The estimator reads the angles it is given as they are: it has no slack constraint, so the angle frame of its output is the frame of its input.

Hybrid multi-rate scheduler¶

HybridMultiRateScheduler.Schedule(start, end, scadaPeriod, pmuPeriod) returns the ordered list of ScheduledUpdate(Time, IsScada) over [start, end] inclusive: a SCADA update at start + k * scadaPeriod and a PMU update at start + k * pmuPeriod. At a coincident instant only the SCADA update is emitted. A non-positive period throws ArgumentException; end < start returns an empty list; the times carry start.Kind. The result is a pure function of its arguments. The class is internal; the intended consumer runs a robust WLS at each SCADA update and the linear estimator at each PMU update, and no such consumer exists in src/.

Horizon guard, as applied¶

ForwardInferenceGuard.RejectFuture(stamp, utcNow, what) throws ForwardInferenceException when stamp is newer than utcNow - HorizonHours, with HorizonHours = 169.0 (7 days plus 1 hour; src/GridSim.Core/Diagnostics/ForwardInferenceGuard.cs). It is applied three times on the PMU path. pmu-ingest calls it on every decoded DATA frame's time against the real clock before anything is printed or written: one frame newer than the horizon refuses the whole capture (REJECTED (forward inference), exit 2), live or recorded. PmuMeasurementSetMapper.Map calls it again on frame.TimeUtc with the label "PMU superframe" before reading anything else (utcNow defaults to DateTime.UtcNow). A MeasurementSet built by the mapper carries TakenUtc = frame.TimeUtc, so MeasurementSet.Load (the reader pmu-estimate and estimate use) applies its own guard when the set is read back. The decoder, the config parser and the PDC do not call it.

Sample quality¶

C37118SampleQuality.Of(frameTimeQuality, stat) gives each PMU block of a DATA frame a C37118SampleVerdict, from IEEE C37.118.2-2011 as the decoder records it ([VERIFY] on the table numbers):

Verdict	Rule
`UnreliableFrameTime`	the FRACSEC message time-quality code (low four bits of the top FRACSEC byte) is above `0x5` (time error 100 us or more, an undefined code `0xC` to `0xE`, or clock failure `0xF`); every block of the frame
`InvalidData`	STAT bits 15-14 are not `00` (PMU error, test mode or absent data)
`Unsynchronised`	STAT bit 13 set, a STAT time quality (bits 8-6) of `100` (100 us) or worse, or an unlocked time (bits 5-4) other than `00`
`Usable`	none of the above

The line is 10 us: a time error dt rotates a 50 Hz phasor by 360 * 50 * dt degrees, 0.18 degrees at 10 us and 1.8 degrees at 100 us, against the 0.57 degrees of angle a 1 % total vector error allows. The other STAT bits (sorting, trigger, configuration change, data modified) do not decide usability. pmu-ingest aligns and maps only Usable samples.

The `pmu-ingest` verb¶

gridsim pmu-ingest (<capture.bin> | --live tcp://host:port|udp://[bind]:port --live-frames N [--live-seconds S]
                    [--live-idcode ID] [--live-quarantine <dir>])
                   [--channels <map.json> --measurements-dir <dir>]

The flags are in the CLI reference (pmu-ingest); the result-file flags (--json, --csv, --out) apply as for other verbs. --channels and --measurements-dir go together (one alone exits 2). The --live-* flags qualify --live and are refused without it (exit 2), and --live takes no capture file.

Input. A capture file holds frames as sent, back to back: CONFIG-1 or CONFIG-2 frames and DATA frames. A capture named *.quarantine.c37, or one with a .quarantine.json sidecar beside it, is refused (exit 1): it is a quarantined live capture and is never read as a recorded one. With --live the receiver (C37118LiveReceiver) turns a device's stream into the same bytes: over TCP GridSim is the client, sends "send CONFIG-2" and "turn on transmission" COMMAND frames to --live-idcode (default 1), frames the stream fail-loud and sends "turn off transmission" when it stops; over UDP it listens on the local port, one frame per datagram, and drops and counts a datagram that is not exactly one frame. The receive stops after --live-frames DATA frames or after --live-seconds (default 30, at most 86,400), whichever is first, and prints a live: line with what arrived.

Framing, then decoding. Pass 1 only splits the bytes into frames: at each frame start it requires at least 4 bytes, 0xAA and a FRAMESIZE of at least 16 that fits the remaining bytes, and reads the 14-byte header. A capture that does not split cleanly is refused whole (exit 1) before any frame is decoded, because nothing says where the next frame starts. Pass 2 walks the frames in order:

Frame type	Handling
CONFIG-1 or CONFIG-2	Counted. A bad CRC is counted and the frame skipped. A good frame is parsed; if it differs from the previous configuration in TIME_BASE, PMU count, any IDCODE, FORMAT, channel counts, FNOM or PHUNIT table, the change is counted. A frame that passes its CRC but cannot be parsed (a TIME_BASE of 0 among the refusals) ends the run with exit 1.
DATA	Before any usable CONFIG frame: counted, not decoded. Otherwise decoded with the stream overload against the configuration in force when it arrived (the last good CONFIG frame before it). `Ok` frames are kept with their FRACSEC time quality; `CrcMismatch` and `Malformed` (including a FRACSEC not below TIME_BASE) are counted and dropped. An `ArgumentException` from the decoder ends the run with exit 1.
HEADER, COMMAND, CONFIG-3, other	Counted as "other", not decoded.

With no usable CONFIG frame the run exits 1.

Quarantine. With --live --live-quarantine <dir>, the decoded DATA frames newer than the horizon are held back first: they are written as received, with the CONFIG frames and every frame that carried no trusted time, to pmu-live-<received UTC>.quarantine.c37 in the folder, with a .quarantine.json sidecar (schema gridsim-pmu-quarantine/1) that marks them non-evidential and tags them CuratedPlaceholder, never Recorded. They are removed from the samples, so no later step sees them. A folder at or below the GDA evidence lake (its DataSchema.json) is refused before any receive (exit 1). Without the flag nothing is held back.

Horizon. Every remaining decoded DATA frame's time must be older than the horizon against the real clock; one fresh frame refuses the whole run with REJECTED (forward inference) and exit 2, before anything is printed or written.

Channel map. With --channels, the map (gridsim-pmu-channels/1, PmuChannelMapFile) is loaded before the capture (a bad map exits 1) and checked against every configuration in force for a decoded DATA frame (ValidateAgainst): every mapped PMU must be in it, every phasor index inside that PMU's phasor count, and, when the CONFIG frame carries PHUNIT types, a voltage channel must be a voltage phasor and a current channel a current phasor (PmuConfig.PhasorIsCurrent). A misfit exits 1 and names the CONFIG frame it fails against. The file:

{ "schema": "gridsim-pmu-channels/1", "case": "case9", "baseMva": 100,
  "reference": { "pmu": 101, "phasor": 0, "anchorDeg": 0 },
  "pmus": [ { "pmu": 101,
              "voltages": [ { "phasor": 0, "bus": 1, "baseVolts": 9526.3, "vmSigmaPu": 0.002, "vaSigmaDeg": 0.1 } ],
              "currents": [ { "phasor": 1, "from": 1, "to": 4, "circuit": null, "baseAmps": 2405.6,
                              "iMagSigmaPu": 0.005, "iAngSigmaDeg": 0.2 } ] } ] }

Every field except case, circuit and anchorDeg is required; a wrong schema, a missing, non-finite or non-positive base or sigma, a negative phasor index, a PMU listed twice, a phasor mapped twice, no channel at all and a reference that is not a mapped voltage channel are refused, every problem named.

Quality and alignment. Each block of each kept frame gets its C37118SampleVerdict; only Usable samples go to a PhasorDataConcentrator built from the IDCODEs of every configuration in force, with a zero wait window and one FlushAll. A dropped sample leaves its PMU missing at that instant. An instant is incomplete when a PMU of the configuration in force then did not report usably.

Measurement sets. With the channel map, every aligned instant goes through PmuMeasurementSetMapper.Map (the horizon again) and is written with SaveWithoutOverwrite as pmu-<yyyyMMddTHHmmssfffffffZ>.json in the folder (a clash gets a -dupN name). An instant whose reference channel did not report usably, or with no mapped PMU, writes no set and is counted.

Report. A banner with the byte count and [EXPERIMENTAL]; with --live, the live: line first; the frame-count line (CONFIG and bad-CRC CONFIG; DATA decoded, bad CRC, malformed, before any CONFIG; other); with the quarantine, a quarantine: line; the configuration line (PMU count, TIME_BASE, DATA_RATE of the last configuration, and, when it changed, the count and that each frame was decoded with the configuration in force); one row per PMU of every configuration in force (| PMU | IDCODE | phasors | samples | first (UTC) | last (UTC) | freq min Hz | freq max Hz | STAT non-zero | dropped |, a phasor count that changed printed as 2/3); the aligned instants and how many have a PMU missing; the quality: line with the dropped samples by reason; with the channel map, the measurement sets: line.

Result files. summary holds file, frames, config_frames, data_decoded, data_bad_crc, data_malformed, data_before_config, other_frames, pmus, time_base, data_rate, config_changes, aligned_instants, incomplete_instants, samples_dropped, dropped_invalid_data, dropped_unsynchronised, dropped_unreliable_frame_time; with the quarantine quarantined_data_frames; with the channel map measurement_sets_written, instants_without_reference, instants_without_mapped_pmu. The samples table holds one row per phasor of every decoded block: time_utc (seven fractional digits), pmu_idcode, station, stat, freq_hz, rocof_hz_per_s, phasor, real, imag, magnitude, angle_deg at full double precision, and quality (the verdict).

Exit codes.

Code	Meaning
0	Every DATA frame decoded, no sample dropped, every instant with a reference wrote its set, and a live receive was complete.
1	An input error: the file cannot be read or is a quarantine capture; it does not frame; a header is unreadable; a CONFIG frame passed its CRC but cannot be parsed; the decoder refused the configuration; no usable CONFIG frame; a bad or misfitting channel map; the output folder cannot be created; a live address or transport failure; a quarantine folder inside the evidence lake or a quarantine write failure.
2	Usage (arguments, flags, the flag pairs above) or `REJECTED (forward inference)`.
3	DATA frames with a bad CRC, malformed or before any CONFIG; samples dropped by the quality rule; instants without a usable reference channel; an incomplete live receive (datagrams that were not one frame, or the timeout before the frames asked for); a live receive with no frame. The summary still prints and stderr names each count.

The `pmu-estimate` verb¶

gridsim pmu-estimate <case> --measurements <set.json> [--json|--csv|--out <file>] runs the linear PMU estimate on one gridsim-measurements/1 set (pmu-estimate). The set is loaded (MeasurementSet.Load, so the horizon guard applies; a fresh set is REJECTED, exit 2) and validated against the case (exit 1 on an input error).

PmuMeasurementSetAdapter.FromSet(set, model) pairs the polar rows into rectangular phasors: a bus's VmPu and VaDeg rows form its voltage phasor, and a branch end's IMagPu and IAngDeg rows (same from, to, end and circuit) its current phasor. Every other row, and a magnitude or angle without its partner, is listed as not used with the reason. Two rows of one kind at one place (ambiguous pairing), a current metered at the To end (the linear model holds the from-end current only) and a phasor of magnitude 0 are refused (exit 1). The error model is the first-order (linearised) transform of the polar sigmas at the measured value: with z = m e^(j theta), s_m in pu and s_theta in radians,

var_re = c^2 s_m^2 + m^2 s^2 s_theta^2        var_im = s^2 s_m^2 + m^2 c^2 s_theta^2
cov    = c s (s_m^2 - m^2 s_theta^2)          c = cos theta, s = sin theta       (DetMath sine and cosine)

Its determinant is m^2 s_m^2 s_theta^2, positive for m > 0, so each pair is weighted by its full 2x2 inverse (Sigma = sqrt(var_re), SigmaIm = sqrt(var_im), CovOffDiag = cov).

A set with no voltage or current phasor exits 1. A gain that is singular (the phasors do not determine every bus voltage) prints NOT OBSERVABLE and exits 3. Otherwise the verb prints the phasors used and not used, J over 2 * (voltages + currents) - 2n degrees of freedom (one non-iterative solve, no bad-data test) and one row per bus (|V| to six decimals, angle to four), exit 0. The result file holds case, measurements, voltage_phasors, current_phasors, rows_not_used, objective_j, degrees_of_freedom, a buses table (bus_id, vm_pu, va_deg) and a not_used table. A set written by pmu-ingest is read by pmu-estimate directly.

Parameters and defaults¶

Name	Value	Unit	Meaning	Source symbol
SYNC byte 0	`0xAA`	-	Start of every frame.	`C37118CommonHeader.Parse`
Common header	14	bytes	Header length.	`C37118CommonHeader.Parse`
Smallest CONFIG frame	24	bytes	Header, TIME_BASE, NUM_PMU, DATA_RATE, CHK with no PMU.	`C37118ConfigParser.MinimumConfigFrameBytes`
Smallest frame `pmu-ingest` splits	16	bytes	Floor on FRAMESIZE when framing a file.	`RunPmuIngest` in `src/GridSim.Cli/Program.cs`
CRC polynomial, initial value	`0x1021`, `0xFFFF`	-	CRC-CCITT, no reflection, no final XOR.	`C37118Crc.Compute`
FRACSEC and TIME_BASE mask	`0x00FFFFFF`	-	Counts of 1/TIME_BASE s; top FRACSEC byte is time quality.	`C37118CommonHeader.Parse`, `C37118ConfigParser.Parse`
FORMAT mask	`0x0F`	-	Four defined bits.	`C37118ConfigParser.Parse`
PHUNIT scale	`(PHUNIT & 0x00FFFFFF) / 100000`	V or A per count	Integer phasor scale.	`PmuConfig.PhasorScale`
Integer polar angle	`1e-4`	rad per count	Signed 16-bit angle.	`C37118Decoder.ReadBlock`
Integer FREQ	`NominalHz + count / 1000`	Hz	Deviation from FNOM in millihertz.	`C37118Decoder.ReadBlock`
Integer DFREQ	`count / 100`	Hz/s	ROCOF times 100.	`C37118Decoder.ReadBlock`
Nominal frequency	50 if FNOM bit 0 is set, else 60	Hz	`PmuConfig.NominalHz`.	`PmuConfig.Is50Hz`
PDC wait window	constructor argument; `pmu-ingest` passes `TimeSpan.Zero`	s	Delay past an instant before emitting it.	`PhasorDataConcentrator`
Horizon	169	h	Newest age `pmu-ingest`, the mapper and `MeasurementSet.Load` admit.	`ForwardInferenceGuard.HorizonHours`
Largest TIME_BASE	`2^24 - 1`	counts per second	A 24-bit field; 0 is refused.	`C37118Decoder.MaxTimeBase`
Usable frame time quality	`0x0` to `0x5`	code	Time within 10 us of UTC.	`C37118SampleQuality.MaxUsableFrameTimeQualityCode`
Usable STAT time quality	`000` to `011`	code	Within 10 us (`000` means not used).	`C37118SampleQuality.MaxUsableStatTimeQuality`
Live receive time limit	30, at most 86,400	s	`--live-seconds`.	`ReceiveLivePmu` in `Program.cs`
Live COMMAND IDCODE	1	-	`--live-idcode`, TCP only.	`ReceiveLivePmu` in `Program.cs`
Quarantine tag	`CuratedPlaceholder`	-	Provenance of every quarantined frame.	`PmuLiveQuarantine.Tag`
Mapper `baseMva`, `caseName`	100.0, `"pmu"`	MVA	Fields of the built set.	`PmuMeasurementSetMapper.Map`
`PmuAngleReference.AnchorDeg`	0	deg	Slack-frame angle of the reference bus.	`PmuAngleReference`
Estimator pivot floor	`1e-12 * max\|G_ij\|`	-	Singular-gain test.	`NumericTolerances.RelativePivotFloor`

Invariants¶

Invariant	Test
CRC-CCITT of `"123456789"` is `0x29B1`.	`C37118DecoderTests.Crc_ccitt_matches_the_standard_check_vector`
A flipped byte fails the CRC, and the decoder reports `CrcMismatch`.	`C37118DecoderTests.A_flipped_byte_fails_the_crc`, `N061PmuIngestTests.A_frame_whose_length_does_not_match_the_configuration_is_malformed_and_a_bad_crc_is_reported`
A float DATA frame round-trips: time, phasors, FREQ, DFREQ.	`C37118DecoderTests.A_data_frame_round_trips_through_the_decoder`
The FORMAT word, written as a raw number, decodes by the standard bit positions (including `0x000E`).	`C37118FormatBitOrderTests.The_raw_format_word_decodes_by_the_standard_bit_positions`, `C37118FormatBitOrderTests.A_float_rectangular_frame_decodes_correctly_with_the_parsed_format`
The parser reads TIME_BASE, DATA_RATE and each PMU's FORMAT.	`C37118ConfigParserTests.Parses_time_base_data_rate_and_per_pmu_format`
The parser keeps PHUNIT and FNOM verbatim.	`C37118IntegerScalingTests.The_config_parser_keeps_phunit_and_fnom`
Integer rectangular and polar phasors and integer FREQ/DFREQ are scaled from the parsed configuration (50 and 60 Hz, unsigned polar magnitude, angle in 1e-4 rad).	`C37118IntegerScalingTests.Integer_rectangular_phasors_and_frequency_are_scaled_from_the_parsed_config`, `C37118IntegerScalingTests.Integer_polar_phasors_scale_the_unsigned_magnitude_and_keep_the_angle_in_1e4_rad_at_60_hz`, `C37118FixedFormatTests.Decodes_a_fixed_integer_rectangular_data_frame`
Float data decodes identically through either overload.	`C37118IntegerScalingTests.Float_data_decodes_the_same_through_either_overload`
An integer-phasor configuration without a PHUNIT table is refused, never returned as raw counts.	`C37118IntegerScalingTests.An_integer_phasor_config_without_phunit_is_refused`, `N061PmuIngestTests.Integer_phasors_without_a_phunit_table_are_refused`
The frame-length check counts analogue channels.	`C37118IntegerScalingTests.The_frame_length_check_counts_the_analogue_channels`
A multi-PMU frame decodes every block in CONFIG order, with each block's own STAT and scaling.	`N061PmuIngestTests.A_two_pmu_frame_decodes_both_blocks_in_config_order`
A one-PMU stream decodes bit for bit as the single-PMU overload (time, STAT, frequency, ROCOF, every phasor component).	`N061PmuIngestTests.A_one_pmu_stream_decodes_bit_for_bit_as_the_single_pmu_overload`
A frame whose length does not match the configuration is `Malformed`.	`N061PmuIngestTests.A_frame_whose_length_does_not_match_the_configuration_is_malformed_and_a_bad_crc_is_reported`
A good CONFIG frame parses every field; bytes after the declared frame are ignored.	`N163C37118ConfigParserRefusalTests.A_good_frame_parses_every_field_as_before`, `N163C37118ConfigParserRefusalTests.Bytes_after_the_declared_frame_are_ignored_as_before`
A CONFIG frame with a wrong CRC, a corrupted body byte, a truncation, a FRAMESIZE below the minimum, a phasor or PMU count that runs past the frame, or no room for DATA_RATE is refused with a named `C37118FrameException`, never a raw range exception.	`N163C37118ConfigParserRefusalTests.A_wrong_crc_is_refused_naming_both_check_words`, `A_corrupted_body_byte_is_refused_by_the_crc`, `A_frame_cut_short_of_its_declared_size_is_refused_as_truncated`, `A_declared_size_below_the_smallest_config_frame_is_refused`, `A_phasor_count_that_runs_past_the_frame_is_refused_by_name_not_by_a_raw_range_exception`, `A_pmu_count_larger_than_the_blocks_present_is_refused_naming_the_missing_block`, `A_frame_with_no_room_for_data_rate_is_refused`
`C37118FrameException` is an `ArgumentException`.	`N163C37118ConfigParserRefusalTests.The_named_error_is_still_an_argument_exception_for_existing_callers`
The PDC aligns PMUs onto common instants in time order.	`PhasorDataConcentratorTests.Aligns_pmus_onto_common_instants_in_time_order`
A PMU that did not report is listed as dropped.	`PhasorDataConcentratorTests.Flags_a_missing_pmu_as_dropped`
An instant is held until it falls behind the wait window.	`PhasorDataConcentratorTests.Holds_an_instant_until_it_falls_behind_the_wait_window`
A straggler for an emitted instant is discarded and counted, never re-emitted; later instants still flow.	`C37118FormatBitOrderTests.A_straggler_for_a_flushed_instant_is_discarded_and_counted_not_re_emitted`, `C37118FormatBitOrderTests.Instants_after_the_high_water_mark_still_flow_normally`
The mapper builds per-unit voltage and current rows at the right bus and branch, with `TakenUtc` from the superframe.	`PmuMeasurementSetMapperTests.Maps_voltage_and_current_phasors_to_measurement_rows`
A dropped PMU contributes no rows.	`PmuMeasurementSetMapperTests.A_dropped_pmu_contributes_no_rows`
A superframe newer than the horizon is refused, including when no angle reference is given.	`PmuMeasurementSetMapperTests.A_fresh_superframe_is_refused_by_the_guard`, `PmuAngleReferenceTests.The_horizon_guard_still_runs_first`
Angles are differences from the reference channel; a common UTC rotation cancels; the difference wraps into +/-180 degrees; the anchor moves a non-slack reference into the slack frame.	`PmuAngleReferenceTests.Angles_are_differences_from_the_reference_channel`, `A_common_utc_frame_rotation_cancels`, `The_difference_is_wrapped_into_plus_minus_180_degrees`, `The_anchor_puts_a_non_slack_reference_into_the_slack_frame`
No angle row is emitted without a reference: no named reference, a dropped reference PMU and an unmapped or non-voltage reference all fail loud.	`PmuAngleReferenceTests.No_named_reference_is_refused_when_there_are_angles_to_emit`, `A_dropped_reference_pmu_is_refused_not_emitted_absolute`, `A_reference_that_is_not_a_mapped_voltage_channel_is_refused`
A full consistent phasor set recovers the solved truth of case9 in one linear solve (`ObjectiveJ < 1e-10`, `Vm` to `1e-9`, `VaDeg` to `1e-7`); a voltage-only set with correlated covariance recovers `Vm` to `1e-9` (its angles and `ObjectiveJ` are not asserted).	`LinearPmuStateEstimatorTests.Full_phasor_set_recovers_the_truth_in_one_linear_solve`, `LinearPmuStateEstimatorTests.Correlated_measurement_covariance_still_recovers_the_truth`
A voltage-only set on case14 recovers every magnitude in one solve; current phasors anchor the angles of unmeasured buses.	`LinearPmuStateEstimatorTests.A_voltage_only_set_on_case14_recovers_every_magnitude_in_one_solve`, `Recovers_angles_at_unmeasured_buses_from_current_anchoring`
The estimator refuses an unknown bus, an unusable sigma or covariance and a non-finite value, naming every problem.	`Doc15SynchrophasorTests.D109_a_voltage_at_a_bus_the_model_does_not_have_is_refused_by_name`, `D109_an_unusable_sigma_or_covariance_is_refused_by_name`, `D109_a_non_finite_value_is_refused_and_every_problem_is_named_at_once`
Frame times are exact integer ticks, floored; TIME_BASE 0 is refused; a FRACSEC not below TIME_BASE is malformed.	`Doc15SynchrophasorTests.D110_instants_the_double_put_one_tick_short_are_now_exact`, `D110_the_fraction_is_floored_to_whole_ticks`, `D111_a_config_frame_with_time_base_0_is_refused_by_name`, `D111_every_decode_overload_refuses_an_impossible_time_base`, `D111_a_fracsec_not_below_time_base_makes_the_frame_malformed`
The mapper bounds-checks every channel and names every misfit.	`Doc15SynchrophasorTests.D008_a_non_reference_channel_past_the_reported_phasors_is_refused_by_name`, `D008_every_bad_channel_is_named_at_once_and_a_pmu_that_did_not_report_is_not_checked`
`pmu-ingest` refuses a fresh capture; drops invalid and unsynchronised samples and exits 3; writes one guarded set per usable instant; refuses a misfitting or lone map; checks the map against every configuration; frames before decoding.	`Doc3CliHorizonProcessTests.D107_pmu_ingest_refuses_a_capture_with_a_frame_newer_than_the_horizon`, `D108_an_invalid_or_unsynchronised_sample_is_dropped_counted_and_exits_3`, `D108_a_good_sample_with_informational_flags_is_used`, `D107_pmu_ingest_with_a_channel_map_writes_one_guarded_measurement_set_per_usable_instant`, `D107_pmu_ingest_refuses_a_channel_map_that_does_not_fit_or_is_given_alone`, `D107_an_instant_whose_reference_channel_was_dropped_writes_no_set_and_exits_3`; `Doc15PmuIngestProcessTests.D008_a_map_that_fits_only_the_later_configuration_is_refused_naming_the_earlier_one`, `D223_a_capture_that_does_not_frame_is_refused_before_any_frame_is_decoded`
Live receive over TCP stops at the frames asked for, refuses bad framing, keeps a bad-CRC frame for the pipeline, reports a timeout; the live run uses the recorded pipeline.	`Cf5LiveC37118Tests`
The quarantine keeps frames byte for byte, never overwrites, is tagged below `Recorded`, refuses the evidence lake, and a quarantine capture is refused as a recorded capture.	`Cf6LivePmuQuarantineTests`
`pmu-estimate`: the adapter pairs rows and lists the rest, the covariance is the first-order polar transform exactly, ambiguous and to-end rows are refused, exact phasors recover the solved state, a set written by `pmu-ingest` is read.	`PmuEstimateTests`
The scheduler interleaves SCADA and PMU updates in strictly increasing time, SCADA winning a coincident instant (5 s and 0.5 s over 5 s: 2 SCADA and 9 PMU updates).	`HybridMultiRateSchedulerTests.Interleaves_scada_and_pmu_with_nine_pmu_per_scada_cycle`
`pmu-ingest` decodes a clean capture and exits 0; counts bad frames and exits 3; refuses an unframeable or CONFIG-less file with exit 1 and a missing argument with exit 2.	`N061PmuIngestTests.Pmu_ingest_decodes_a_clean_capture_and_exits_0`, `Pmu_ingest_counts_bad_frames_and_exits_3`, `Pmu_ingest_refuses_a_file_it_cannot_frame_or_with_no_config`
`pmu-ingest` writes the shared result files.	`CliResultWriterTests` (the verb is in its coverage list)
The experimental markers are present in the PDC, mapper and estimator sources.	`R062EstimationWiringTests.Stale_docs_are_corrected_and_test_only_classes_are_marked_experimental`

The decode and alignment path is single-threaded scalar arithmetic with no parallel step. The linear estimator is tested against a tolerance, not bit for bit, and no test compares its output across CPU tiers. The inverse kernel it uses, DseLinAlg.Invert, has its own bit-identity test (KalmanDenseKernelBitIdentityTests).

Worked example¶

The example uses the two-PMU stream of N061PmuIngestTests. Its frames are rebuilt byte for byte by docs/_captures/c37118/make-captures.py into docs/_captures/c37118/two-pmu.bin: a CONFIG-2 frame and three DATA frames 20 ms apart. STN_A (IDCODE 101) sends two float rectangular phasors with float FREQ/DFREQ (FORMAT 0x000A). STN_B (IDCODE 102) sends one integer rectangular phasor, one analogue value and one digital word, with PHUNIT 1,000,000, which is 10 V per count.

The CONFIG frame, by hand¶

The CONFIG frame is 424 bytes. Its leading fields and the two PMU headers read:

AA 31 01 A8 00 07 65 53 F1 00 00 00 00 00      common header
00 0F 42 40  00 02                              TIME_BASE, NUM_PMU
"STN_A" + 11 spaces  00 65  00 0A  00 02 00 00 00 00      STN, IDCODE, FORMAT, PHNMR, ANNMR, DGNMR
"STN_B" + 11 spaces  00 66  00 00  00 01 00 01 00 01      STN, IDCODE, FORMAT, PHNMR, ANNMR, DGNMR

AA 31: SYNC 0xAA; 0x31 is type 3 (Config2), version 1.
01 A8 is FRAMESIZE 424; 00 07 is IDCODE 7; 65 53 F1 00 is SOC 1,700,000,000; FRACSEC is 0.
00 0F 42 40 is TIME_BASE 1,000,000; 00 02 is NUM_PMU 2.
STN_A: IDCODE 00 65 is 101; FORMAT 00 0A is binary 1010, so PhasorFloat (bit 1) and FreqFloat (bit 3), rectangular; PHNMR 2, no analogues, no digital words. Its PHUNIT words are 00 00 00 00 and 01 00 00 00 (a current channel, factor 0), which float data never uses.
STN_B: IDCODE 00 66 is 102; FORMAT 00 00 is all integer, rectangular; PHNMR 1, ANNMR 1, DGNMR 1. Its PHUNIT word is 00 0F 42 40 (voltage, factor 1,000,000 times 1e-5 gives 10 V per count). FNOM is 00 01 for both PMUs (bit 0 set: 50 Hz).
The last four bytes are DATA_RATE 00 32 (50) and CHK 57 26.

The first DATA frame, by hand¶

The first DATA frame is 56 bytes: 14 + (2 + 2*8 + 8) + (2 + 4 + 4 + 2 + 2) + 2.

AA 01 00 38 00 07 65 53 F1 01 00 00 00 00      common header
00 00                                          STN_A  STAT
48 60 9C 00  C4 7A 00 00                       phasor 0: real, imag (float32)
42 F1 00 00  41 F2 00 00                       phasor 1: real, imag (float32)
42 48 0A 3D  BC A3 D7 0A                       FREQ, DFREQ (float32)
80 00                                          STN_B  STAT
08 FC  FF 9C                                   phasor 0: real, imag (int16)
00 0F  FF FC                                   FREQ, DFREQ (int16)
00 00  00 00                                   analogue, digital word (stepped over)
4D 2A                                          CHK

Header: 0x01 is type 0 (Data), version 1; 00 38 is 56 bytes. SOC is 0x6553F101 = 1,700,000,001 s, which is 2023-11-14T22:13:21Z; FRACSEC is 0, so TimeUtc is that instant exactly.
STN_A, float phasor 0: 48 60 9C 00 is 230000.0 and C4 7A 00 00 is -1000.0. Check the second by hand: sign 1, exponent 0x88 = 136, so the scale is 2^(136-127) = 512, mantissa 0x7A0000 / 2^23 = 0.953125, and -1.953125 * 512 = -1000. Phasor 1 is 42 F1 00 00 = 120.5 and 41 F2 00 00 = 30.25. Then |V0| = sqrt(230000^2 + 1000^2) = 230002.17... and angle = atan2(-1000, 230000) = -0.2491... degrees.
STN_A, FREQ 42 48 0A 3D: the float32 nearest 50.01, which as a double is 50.0099983215332. DFREQ BC A3 D7 0A is the float32 nearest -0.02.
STN_B: STAT 80 00 is 0x8000 (32768). Phasor 0 is 08 FC = 2300 counts and FF 9C = -100 counts; scale 1,000,000 / 100000 = 10, so the phasor is 23000 - j1000 volts. FREQ 00 0F = 15, so 50 + 15/1000 = 50.015 Hz; DFREQ FF FC = -4, so -4/100 = -0.04 Hz/s.
CHK = 0x4D2A is the CRC-CCITT of the 54 bytes before it; the decoder accepts the frame.

The frame length is exactly what the configuration demands, so the stream overload returns Ok.

The captured run¶

The command and its output are in docs/_captures/c37118/pmu-ingest-clean.txt. Timings do not appear in the output, but the timestamps and the byte count are those of the capture file and differ for a different capture.

$ gridsim pmu-ingest docs/_captures/c37118/two-pmu.bin
PMU INGEST - docs/_captures/c37118/two-pmu.bin  [IEEE C37.118.2 capture, 592 bytes]   [EXPERIMENTAL]
  4 frame(s): 1 CONFIG (0 bad CRC), 3 DATA (3 decoded, 0 bad CRC, 0 malformed, 0 before any CONFIG), 0 other
  configuration: 2 PMU(s), TIME_BASE 1000000, DATA_RATE 50
  | PMU | IDCODE | phasors | samples | first (UTC) | last (UTC) | freq min Hz | freq max Hz | STAT non-zero | dropped |
  |-----|--------|---------|---------|-------------|------------|-------------|-------------|---------------|---------|
  | STN_A | 101 | 2 | 3 | 2023-11-14T22:13:21.000000Z | 2023-11-14T22:13:21.040000Z | 50.0100 | 50.0100 | 0 | 0 |
  | STN_B | 102 | 1 | 3 | 2023-11-14T22:13:21.000000Z | 2023-11-14T22:13:21.040000Z | 50.0150 | 50.0150 | 3 | 3 |
  aligned instants: 3 (3 with a PMU missing)
  quality: 3 sample(s) dropped and not used (3 invalid data (STAT bits 15-14), 0 unsynchronised (STAT bit 13, time quality or unlocked time), 0 with an unreliable frame time (FRACSEC time quality))
pmu-ingest: 3 PMU sample(s) dropped by the STAT / time-quality rule (3 invalid data, 0 unsynchronised, 0 unreliable frame time); they are counted, not used.
[exit 3]

The file is 592 bytes: 424 for the CONFIG frame and 3 * 56 = 168 for the DATA frames. The three DATA frames are 20 ms apart (FRACSEC counts 0, 20,000 and 40,000 at TIME_BASE 1,000,000), which gives the first and last times. STN_B shows 3 non-zero STAT words because its STAT is 0x8000 in every frame: STAT bits 15-14 are 10 (test mode or absent data), so the quality rule gives each STN_B sample the verdict InvalidData, drops it from the alignment and the run exits 3 with the dropped column and the quality: line counting the three samples.

The decoded values¶

--csv writes every phasor of every decoded block at full precision, with the sample's quality verdict in the last column; a dropped sample is listed with its verdict, not left out. The run is captured in docs/_captures/c37118/pmu-ingest-csv.txt, and the table is docs/_captures/c37118/samples.csv.

$ gridsim pmu-ingest docs/_captures/c37118/two-pmu.bin --csv docs/_captures/c37118/samples.csv
PMU INGEST - docs/_captures/c37118/two-pmu.bin  [IEEE C37.118.2 capture, 592 bytes]   [EXPERIMENTAL]
  4 frame(s): 1 CONFIG (0 bad CRC), 3 DATA (3 decoded, 0 bad CRC, 0 malformed, 0 before any CONFIG), 0 other
...
  quality: 3 sample(s) dropped and not used (3 invalid data (STAT bits 15-14), 0 unsynchronised (STAT bit 13, time quality or unlocked time), 0 with an unreliable frame time (FRACSEC time quality))
pmu-ingest: 3 PMU sample(s) dropped by the STAT / time-quality rule (3 invalid data, 0 unsynchronised, 0 unreliable frame time); they are counted, not used.
[pmu-ingest] wrote docs\_captures\c37118\samples.csv, docs\_captures\c37118\samples-summary.csv
[exit 3]

The first three rows of samples.csv, the first DATA frame:

time_utc,pmu_idcode,station,stat,freq_hz,rocof_hz_per_s,phasor,real,imag,magnitude,angle_deg,quality
2023-11-14T22:13:21.0000000Z,101,STN_A,0,50.0099983215332,-0.019999999552965164,0,230000,-1000,230002.1739027699,-0.2491105151529136,Usable
2023-11-14T22:13:21.0000000Z,101,STN_A,0,50.0099983215332,-0.019999999552965164,1,120.5,30.25,124.23893310874816,14.092169052423335,Usable
2023-11-14T22:13:21.0000000Z,102,STN_B,32768,50.015,-0.04,0,23000,-1000,23021.728866442678,-2.489552921999156,InvalidData

These match the hand decode: STN_A phasor 0 is 230000 - j1000 V with magnitude 230002.1739... V and angle -0.2491... degrees; STN_B phasor 0 is 23000 - j1000 V (2300 and -100 counts at 10 V per count), frequency 50.015 Hz and ROCOF -0.04 Hz/s. The float32 quantisation of STN_A shows in the frequency 50.0099983215332 and the ROCOF -0.019999999552965164, which are exactly the stored float32 values converted to double.

A damaged capture¶

docs/_captures/c37118/two-pmu-damaged.bin is the same stream with a DATA frame in front of the CONFIG frame and a flipped byte in its last DATA frame. The run is captured in docs/_captures/c37118/pmu-ingest-damaged.txt.

$ gridsim pmu-ingest docs/_captures/c37118/two-pmu-damaged.bin
PMU INGEST - docs/_captures/c37118/two-pmu-damaged.bin  [IEEE C37.118.2 capture, 648 bytes]   [EXPERIMENTAL]
  5 frame(s): 1 CONFIG (0 bad CRC), 4 DATA (2 decoded, 1 bad CRC, 0 malformed, 1 before any CONFIG), 0 other
  configuration: 2 PMU(s), TIME_BASE 1000000, DATA_RATE 50
  | PMU | IDCODE | phasors | samples | first (UTC) | last (UTC) | freq min Hz | freq max Hz | STAT non-zero | dropped |
  |-----|--------|---------|---------|-------------|------------|-------------|-------------|---------------|---------|
  | STN_A | 101 | 2 | 2 | 2023-11-14T22:13:21.000000Z | 2023-11-14T22:13:21.020000Z | 50.0100 | 50.0100 | 0 | 0 |
  | STN_B | 102 | 1 | 2 | 2023-11-14T22:13:21.000000Z | 2023-11-14T22:13:21.020000Z | 50.0150 | 50.0150 | 2 | 2 |
  aligned instants: 2 (2 with a PMU missing)
  quality: 2 sample(s) dropped and not used (2 invalid data (STAT bits 15-14), 0 unsynchronised (STAT bit 13, time quality or unlocked time), 0 with an unreliable frame time (FRACSEC time quality))
pmu-ingest: 2 DATA frame(s) could not be decoded (1 bad CRC, 0 malformed, 1 before any CONFIG); they are counted, not used.
pmu-ingest: 2 PMU sample(s) dropped by the STAT / time-quality rule (2 invalid data, 0 unsynchronised, 0 unreliable frame time); they are counted, not used.
[exit 3]

The two bad frames are counted and left out of the table; the two good frames are decoded and aligned, and their STN_B samples are dropped by the quality rule as in the first run; the exit code is 3. A bad frame never produces a zeroed or partial sample.

A truncated capture¶

docs/_captures/c37118/two-pmu-truncated.bin is two-pmu.bin with its last 10 bytes cut off. The run is captured in docs/_captures/c37118/pmu-ingest-truncated.txt.

$ gridsim pmu-ingest docs/_captures/c37118/two-pmu-truncated.bin
pmu-ingest: docs/_captures/c37118/two-pmu-truncated.bin: the frame at byte 536 declares 56 bytes but only 46 remain; the capture is truncated or corrupt.
[exit 1]

The two frames before the bad one are read and then discarded; the file is refused and no summary prints. The usage case, captured in docs/_captures/c37118/pmu-ingest-usage.txt:

$ gridsim pmu-ingest
usage: gridsim pmu-ingest (<capture.bin> | --live tcp://host:port|udp://[bind]:port --live-frames N [--live-seconds S] [--live-idcode ID] [--live-quarantine <dir>])
                          [--channels <map.json> --measurements-dir <dir>]   (an IEEE C37.118.2 stream: CONFIG then DATA frames)
[exit 2]

The mapper on the same numbers¶

pmu-ingest runs the mapper when given a channel map; this step is arithmetic on the fixture of PmuMeasurementSetMapperTests.Maps_voltage_and_current_phasors_to_measurement_rows. PMU 1 reports a voltage phasor of 102 V at 0.1 rad, mapped to bus 4 with BaseVolts = 100, and a current of 5 A at -0.2 rad on branch 4 to 5 with BaseAmps = 10. PMU 2 reports 100 V at 0.05 rad at bus 1 and is the named reference (PmuAngleReference(PmuId: 2, PhasorIndex: 0)). The rows are:

bus 4  VmPu    = 102 / 100            = 1.02
bus 4  VaDeg   = (0.1 - 0.05) rad     = 0.05 * 180/pi    = 2.8648 degrees
4->5   IMagPu  = 5 / 10               = 0.5
4->5   IAngDeg = (-0.2 - 0.05) rad    = -0.25 * 180/pi   = -14.3239 degrees
bus 1  VaDeg   = 0 (the reference against itself)

The test asserts these to six decimals, the reference row to exactly 0, and asserts TakenUtc equals the superframe time, 174 hours before its utcNow (HorizonHours + 5). The same superframe stamped 1 hour before utcNow raises ForwardInferenceException.

Source map¶

File	Type	Responsibility
`src/GridSim.Core/Synchrophasor/C37118Frame.cs`	`C37118FrameType`, `C37118DataFormat`, `Phasor`, `C37118CommonHeader`, `C37118Crc`	Frame type and FORMAT enums, rectangular phasor value, 14-byte header parse, CRC-CCITT.
`src/GridSim.Core/Synchrophasor/C37118ConfigFrame.cs`	`PmuConfig`, `C37118ConfigFrame`, `C37118FrameException`, `C37118ConfigParser`	CONFIG-1/2 parse with ordered, bounded refusals; PHUNIT scale and channel type; FNOM.
`src/GridSim.Core/Synchrophasor/C37118Decoder.cs`	`C37118Decoder`, `FrameDecodeStatus`, `C37118DataFrame`, `C37118PmuBlock`, `C37118StreamDataFrame`	CRC-first DATA decode, three overloads, block sizing, integer scaling, time conversion.
`src/GridSim.Core/Synchrophasor/PhasorDataConcentrator.cs`	`PmuSample`, `PdcSuperframe`, `PhasorDataConcentrator`	Alignment on reporting instants, wait window, dropouts, late-sample counting.
`src/GridSim.Core/Synchrophasor/PmuMeasurementSetMapper.cs`	`PmuVoltageChannel`, `PmuCurrentChannel`, `PmuChannelMap`, `PmuAngleReference`, `PmuMeasurementSetMapper`	Superframe to `MeasurementSet` rows, angle re-referencing, horizon guard.
`src/GridSim.Core/Estimation/LinearPmuStateEstimator.cs`	`PmuVoltageMeasurement`, `PmuCurrentMeasurement`, `LinearPmuResult`, `LinearPmuStateEstimator`	One-shot rectangular WLS from voltage and current phasors.
`src/GridSim.Core/Estimation/Dse/KalmanStateEstimator.cs`	`DseLinAlg`	`Invert` and `MatVec` used by the linear estimator.
`src/GridSim.Core/Synchrophasor/C37118SampleQuality.cs`	`C37118SampleQuality`, `C37118SampleVerdict`	The STAT and time-quality rule.
`src/GridSim.Core/Synchrophasor/PmuChannelMapFile.cs`	`PmuChannelMapFile`, `PmuChannelMapSet`	`gridsim-pmu-channels/1` load and the check against a CONFIG frame.
`src/GridSim.Core/Synchrophasor/C37118LiveTransport.cs`	`C37118LiveReceiver`, `C37118StreamFramer`, `C37118FrameCheck`, the COMMAND frame	Live TCP and UDP receive.
`src/GridSim.Core/Synchrophasor/PmuLiveQuarantine.cs`	`PmuLiveQuarantine`	The live quarantine capture and sidecar.
`src/GridSim.Core/Estimation/PmuMeasurementSetAdapter.cs`	`PmuMeasurementSetAdapter`, `PmuPhasorSet`	Polar set rows to rectangular phasors with the first-order covariance.
`src/GridSim.Core/Simulation/HybridMultiRateScheduler.cs`	`ScheduledUpdate`, `HybridMultiRateScheduler` (internal)	Interleaved SCADA and PMU update schedule.
`src/GridSim.Core/Diagnostics/ForwardInferenceGuard.cs`	`ForwardInferenceGuard`, `ForwardInferenceException`	The 169 h horizon.
`src/GridSim.Cli/Program.cs`	`RunPmuIngest`, `ReceiveLivePmu`, `RunPmuEstimate`	The two verbs: framing, per-frame handling, quality, alignment, sets, report, result files, exit codes.
`src/GridSim.Cli/CliResultWriter.cs`	verb list	Registers `pmu-ingest` among the verbs that write result files.
`tests/GridSim.Tests/C37118*Tests.cs`, `N061PmuIngestTests.cs`, `N163C37118ConfigParserRefusalTests.cs`, `PhasorDataConcentratorTests.cs`, `PmuAngleReferenceTests.cs`, `PmuMeasurementSetMapperTests.cs`, `LinearPmuStateEstimatorTests.cs`, `HybridMultiRateSchedulerTests.cs`	tests	The invariants above.
`docs/_captures/c37118/`	captures	`make-captures.py`, the three capture inputs and the captured runs.

Limits¶

The Synchrophasor types, the linear estimator and the scheduler are experimental (R062, R095): no resident engine runs the PMU path, and the scheduler has no consumer.
A device streaming the present is refused by the horizon (or, with --live-quarantine, captured and not processed); only a PDC or historian replaying history older than 169 h is ingested live.
The linear estimator forms a dense 2n x 2n gain and an explicit inverse, has no slack constraint and no bad-data test, holds the from-end current only, and is tested to tolerance (1e-9 on Vm), not bit for bit. The adapter's covariance is a first-order linearisation at the measured value.
The scheduler does not check mixed DateTimeKind values among its arguments.
Analogue and digital channels are sized and skipped, not decoded; channel names, ANUNIT and DIGUNIT are skipped; HEADER, COMMAND and CONFIG-3 frames are counted and not decoded.
The frame version in SYNC byte 1 is read and not validated.
The IEEE C37.118.2-2011 text is not in docs/reference; the table numbers cited in the code comments are as earlier work recorded them and carry the provenance tag [VERIFY].