I'm currently looking into some of the inner workings of DOCSIS and related. One thing I'm struggling a bit with is how cable modem config files are made.
From what I've gathered:
CM configs are binary files based on a TLV format.
These configs are deployed using a TFTP server, hinted via DHCP to the modem itself when it boots.
I'm interested in knowing how these config files are structured. I have next to no knowledge of TLV aside from what I've read these past days.
Is TLV just a generic method of stringing data together? It seems like TLV is used in both binary forms and json-like strings of clear text.
Is the T and/or L parts of TLV a set size (in bytes)? How do I know where they separate?
I think I read somewhere that CM configs use a subset of TLV called TLV-BER. If I'm not mistaken, this adds support for TLV nesting. How does this work?
I've heard that DOCSIS is well documented over at CableLabs, but I haven't been able to find this information yet. Helpful links are appreciated.
I have a binary config file in my possession that I've been able to decode using online tools, but assume I want to make my own tool for this purpose, how would I go about it?
Edit: Here's an excerpt of the first 64 bytes of the binary file, for reference.
03010112 01021916 08040668 51E00601 07070104 01020065 09040000 17C8181D
08040203 A0000904 00001F40 0E021F40 01020001 0F010207 01040601 07270101
03 01 01 ; 3 - Network access control: ON
12 01 02 ; 18 - Max Number of CPEs: 2
19 16 ; 25 - Downstream service flow
08 04 06 68 51 E0 ; 25.8 - Maximum Sustained Traffic Rate
06 01 07 ; 25.6 - Quality of Service Parameter Set Type: prov/adm/active
07 01 04 ; 25.7 - Traffic Priority: 4
01 02 00 65 ; 25.1 - Service Flow Reference or ASF Reference: 65
09 04 00 00 17 C8 ; 25.9 - Maximum Traffic Burst: 6,088
18 1D ; 24 - Upstream service flow
08 04 02 03 A0 00 ; 24.8 - Maximum Sustained Traffic Rate: 33,792,000
09 04 00 00 1F 40 ; 24.9 - Maximum Traffic Burst: 8,000
0E 02 1F 40 ; 24.14 - Maximum Concatenated Burst: 8,000
01 02 00 01 ; 24.1 - Service Flow Reference or ASF Reference: 1
0F 01 02 ; 24.15 - Service Flow Scheduling Type: 2
07 01 04 ; 24.7 - Traffic Priority: 4
06 01 07 ; 24.6 - QOS: 0x07 (prov|adm|active)
27 01 01 ; 39 - Enable 2.0 Mode: ON
Types
Type Description Spec Defined
======== ========================================================= ============
0 Pad DOCSIS 1.0
1 Downstream Frequency DOCSIS 1.0
2 Upstream Channel ID DOCSIS 1.0
3 Network Access Control Object DOCSIS 1.0
4 DOCSIS 1.0 Class of Service DOCSIS 1.0
5 Modem Capabilities Encoding DOCSIS 1.0
6 CM Message Integrity Check (MIC) DOCSIS 1.0
7 CMTS Message Integrity Check (MIC) DOCSIS 1.0
8 Vendor ID Encoding DOCSIS 1.0
9 SW Upgrade Filename DOCSIS 1.0
10 SNMP Write Access Control DOCSIS 1.0
11 SNMP MIB Object DOCSIS 1.0
12 Modem IP Address DOCSIS 1.0
13 Service(s) Not Available Response DOCSIS 1.0
14 CPE Ethernet MAC Address DOCSIS 1.0
15 Telephone Settings Option (deprecated) DOCSIS 1.0
17 Baseline Privacy (Security) DOCSIS 1.0
18 Max Number of CPEs DOCSIS 1.0
19 TFTP Server Timestamp DOCSIS 1.0
20 TFTP Server Provisioned Modem IPv4 Address DOCSIS 1.0
21 SW Upgrade IPv4 TFTP Server DOCSIS 1.0
22 Upstream Packet Classification DOCSIS 1.1
23 Downstream Packet Classification DOCSIS 1.1
24 Upstream SF DOCSIS 1.1
25 Downstream SF (11.4) DOCSIS 1.1
26 Payload Header Suppression DOCSIS 1.1
27 HMAC-Digest DOCSIS 3.1
28 Maximum Number of Classifiers DOCSIS 1.1
29 Privacy Enable DOCSIS 1.1
30 Authorization Block DOCSIS 1.1
31 Key Sequence Number DOCSIS 1.1
32 Manufacturer Code Verification Certificate DOCSIS 1.1
33 Co-Signer Code Verification Certificate DOCSIS 1.1
34 SNMPv3 Kickstart Value DOCSIS 1.1
35 Subscriber Mgmt Control DOCSIS 1.1
36 Subscriber Mgmt CPE IPv4 List DOCSIS 1.1
37 Subscriber Mgmt Filter Groups DOCSIS 1.1
38 SNMPv3 Notification Receiver DOCSIS 1.1
39 Enable 2.0 Mode DOCSIS 2.0
40 Enable Test Modes DOCSIS 2.0
41 Downstream Channel List DOCSIS 2.0
42 Static Multicast MAC Address DOCSIS 2.0
43 DOCSIS Extension Field DOCSIS 1.0
44 Vendor Specific Capabilities DOCSIS 1.0
45 Downstream Unencrypted Traffic (DUT) Filtering DOCSIS 2.0
46 Transmit Channel Configuration (TCC) DOCSIS 3.0
47 Service Flow SID Cluster Assignment DOCSIS 3.0
48 Receive Channel Profile DOCSIS 3.0
49 Receive Channel Configuration DOCSIS 3.0
50 DSID Encodings DOCSIS 3.0
51 Security Association Encoding DOCSIS 3.0
52 Initializing Channel Timeout DOCSIS 3.0
53 SNMPv1v2c Coexistence DOCSIS 3.0
54 SNMPv3 Access View Configuration DOCSIS 3.0
55 SNMP CPE Access Control DOCSIS 3.0
56 Channel Assignment Configuration Settings DOCSIS 3.0
57 CM Initialization Reason DOCSIS 3.0
58 SW Upgrade IPv6 TFTP Server DOCSIS 3.0
59 TFTP Server Provisioned Modem IPv6 Address DOCSIS 3.0
60 Upstream Drop Packet Classification DOCSIS 3.0
61 Subscriber Mgmt CPE IPv6 Prefix List DOCSIS 3.0
62 Upstream Drop Classifier Group ID DOCSIS 3.0
63 Subscriber Mgmt Control Max CPE IPv6 Prefix DOCSIS 3.0
64 CMTS Static Multicast Session Encoding DOCSIS 3.0
65 L2VPN MAC Aging Encoding DOCSIS 2.0
66 Management Event Control Encoding DOCSIS 3.0
67 Subscriber Mgmt CPE IPv6 Prefix List DOCSIS 3.0
68 Default Upstream Target Buffer Configuration DOCSIS 3.0
69 MAC Address Learning Control Encoding DOCSIS 3.0
70 Upstream Aggregate Service Flow Encodings DPoE 2.0
71 Downstream Aggregate Service Flow Encodings DPoE 2.0
72 Metro Ethernet Service Profile DPoE 2.0
73 Network Timing Profile DPoE 2.0
74 Energy Management Parameter Encoding DOCSIS 3.0
75 Energy Mgt. Mode Indicator DOCSIS 3.1
76 Energy Mgt. Identifier List for CM DOCSIS 3.1
77 DOCSIS Time Protocol Enable DOCSIS 3.1
78 AQM Disable DOCSIS 3.1
79 UNI Control Encoding DOCSIS 3.0
80 Energy Management – DOCSIS Light Sleep Encodings DOCSIS 3.1
81 Manufacturer CVC Chain DOCSIS 3.1
82 Co-signer CVC Chain DOCSIS 3.1
83 L2CP Management DPoE 2.0
201-231 eCM eSAFE Configuration File TLVs
201 ePS
202 eRouter eRouter
203-215 Reserved
216 eMTA PacketCable 1.x
217 eSTB DSG
218 Reserved
219 eTEA TEI
220 eDVA PacketCable 2.0
221 eSG SMA gateway
222-231 Reserved
255 End-of-Data DOCSIS 1.0
Bonus Reading
CableLabs' Assigned Names and Numbers - CL-SP-CANN-I15-170111 (.pdf)
How to create a cable modem config file 🕗
Your HEX dump does not seems using ASN.1 BER encoding. It is TLV encoding with Tag field 1 Byte length and Length field 1 Byte HEX value.
Tags 0x16, 0x18 and 0x1F seems constructed templates which contains another group of tags.
Meaning of Tag Values lookup in your documentation.
Here is possible example of parsing in T-L-V separation form:
03-01-01
12-01-02
19-16 // template?
08-04-066851E0
06-01-07
07-01-04
01-02-0065
09-04-000017C8
18-1D // template?
08-04-0203A000
09-04-0000
1F-40 // template?
0E-02-1F40
01-02-0001
0F-01-02
07-01-04
06-01-07
27-01-01
...example cut here
Related
using Android phone I need read NDEF message (URI) from a microcontroller based NFC device. I read several standards like ISO/IEC 7816-4:2005, NFC Data Exchange Format (NDEF) and others. For now i have the following data exchange (crc bytes omitted when tx):
RX: 00 a4 04 00 07 d2 76 00 00 85 01 01 00 35 c0 // android selects file1
TX: 90 00
RX: 00 a4 00 0c 02 e1 03 d2 af //android selects file2
TX: 90 00
RX: 00 b0 00 00 0f 8e a6
TX: 00 01 02 03 04 05 06 07 08 09 //it is the dummy data
//here android restarts autocollision detection process
Maybe somebody can explain me the workflow of the commands. I dont know what file id means (file1= "d2 76 00 00 85 01 01" and file2= "e1 03")
I stopped at the first READ_BINARY command. i dont know what to send in response and where to read about this. I supposed that i need somehow insert the NDEF payload here. Also here another nfc topic
i found that the response to the first BO should be so called "capability container" but i dont know what is it, and it is not said in the NDEF specification.
The NFC spec for Type 4 cards is a good read
"d2 76 00 00 85 01 01" is the AID (Application ID) used for NDEF
"e1 03" is the Capability Container File ID
A summary of NDEF reading from a Type 4 card is:-
Select the NDEF AID
Select the Capability Container File ID
Read 15 Bytes of the Capability Container File
Select NDEF file ID
Read NDEF file
Capability Container File contains things like the File ID of the NDEF file, NDEF version number, NDEF read/write security, Maximum NDEF size,Maximum read sizes, etc.
All defined in the link of the NDEF spec I gave.
Here i found the description of the protocol NFC Forum Type 4 Tag IC with 16-Kbit EEPROM
I'm using libnfc and the apdu_examle.c with PN532 on my Beaglebone.
I have a android example for emulating Miffare classic 1k card HCE(Host Card emulation) on my phone and everyting is working fine. If I tap the phone I can read the message that I'm sending from my phone.
Is it possible to send apdu commands to read data from a Miffare classic 1k card (passive tag)? not phone.
The card is ISO/IEC 14443A standard.
Must there be a specific format or data structure on the card?
Here are some more info about the card:
ISO/IEC 14443A (106 kbps) target:
ATQA (SENS_RES): 00 04
UID size: single
bit frame anticollision supported
UID (NFCID1): db 6c 10 2b
SAK (SEL_RES): 08
Not compliant with ISO/IEC 14443-4
Not compliant with ISO/IEC 18092
Fingerprinting based on MIFARE type Identification Procedure:
MIFARE Classic 1K
MIFARE Plus (4 Byte UID or 4 Byte RID) 2K, Security level 1
SmartMX with MIFARE 1K emulation
MIFARE Classic does not operate with APDUs (ISO 7816 layer 4), but at layer 3 + proprietary extensions. As far as I know, it is not possible to communicate with it using HCE unless the phone has an NFC chipset from NXP.
But if your card is a SmartMX with emulated MIFARE, that means that you can send APDUs to the JavaCard OS. Have you tried sending a simple command, for example an empty SELECT 00A4040000?
One has to make a couple of adjustments to run the sample_wave.py example program on Python 3. But having made them and having sent a .wav file to the Voice endpoint I get:
b'HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nTransfer-Encoding: chunked\r\nConnection: close\r\n\r\n'b'e14\r\n{"Format":"SoundHoundVoiceSearchResult","FormatVersion":"1.0","Status":"Error","ErrorMessage":"Could not create speech handler because audio is not supported. (audio bytes received:
The audio bytes that followed the colon start with
\"52 49 46 46 62 27 24 5c 78 30 30 5c 78 30 30 5c 78 ...
Any suggestions?
Thanks in advance.
Cheers, Scott
It sounds like the audio file isn't supported, most likely because you need a header.
You can read more about supported formats here. https://www.houndify.com/docs#compressing-and-streaming-audio-formats
Best,
James
good afternoon
I can not connect the device
Here the data on it:
Full Speed device # 5 (0x14200000): Composite device: "FT240X USB FIFO"
Port Information: 0x001a
Not Captive
Attached to Root Hub
External Device
Connected
Enabled
Number Of Endpoints (includes EP0):
Total Endpoints for Configuration 1 (current): 3
Device Descriptor
Descriptor Version Number: 0x0200
Device Class: 0 (Composite)
Device Subclass: 0
Device Protocol: 0
Device MaxPacketSize: 8
Device VendorID/ProductID: 0x0403/0x6015 (Future Technology Devices International Limited)
Device Version Number: 0x1000
Number of Configurations: 1
Manufacturer String: 1 "FTDI"
Product String: 2 "FT240X USB FIFO"
Serial Number String: 3 "DA6LXJK"
Configuration Descriptor (current config)
Length (and contents): 32
Raw Descriptor (hex) 0000: 09 02 20 00 01 01 00 80 2D 09 04 00 00 02 FF FF
Raw Descriptor (hex) 0010: FF 02 07 05 81 02 40 00 00 07 05 02 02 40 00 00
Unknown Descriptor 0020:
Number of Interfaces: 1
Configuration Value: 1
Attributes: 0x80 (bus-powered)
MaxPower: 90 mA
Interface #0 - Vendor-specific "FT240X USB FIFO"
Alternate Setting 0
Number of Endpoints 2
Interface Class: 255 (Vendor-specific)
Interface Subclass; 255 (Vendor-specific)
Interface Protocol: 255
Endpoint 0x81 - Bulk Input
Address: 0x81 (IN)
Attributes: 0x02 (Bulk)
Max Packet Size: 64
Polling Interval: 0 ms
Endpoint 0x02 - Bulk Output
Address: 0x02 (OUT)
Attributes: 0x02 (Bulk)
Max Packet Size: 64
Polling Interval: 0 ms
when it is connected to the system should appear in the external (USB) audio device on which to send the sound settings
what should I do? write your kext? if so, how? where to start at all?
the manufacturer's website no drivers
A comprehensive answer to this question could probably fill a small book, but I'll try to give you some starting points.
USB Device Class Definition for Audio Devices
If the audio device adheres to the standard USB Audio Device class, but for some reason doesn't advertise this in the standard way, you can probably still use Apple's driver for such devices, by creating a codeless kext, or possibly a very small kext with some helper code.
There are a lot of details to this which Apple documents in TN2274 here.
Nonstandard USB audio
If your device uses a custom USB audio interface, you'll need to write your own driver. The kernel side API for this, IOAudioEngine etc. is actually deprecated since OSX Yosemite, with Apple recommending to write userspace Audio drivers instead. They provide some extensive sample code for getting started with this. Note that USB devices can be driven entirely from user space, so you don't even need a split USB/kernel driver architecture like you would for a PCI audio device.
Recently I decided to look closer at the payload bytes of the tcpdump file format i.e. pcap, and I realized they don't make sense. These are collected on OS/X.
They absolutely always begin with 00 00.
Payload lengths are often around 39 bytes, 310 bytes, 1500 bytes.
Looking at the bytes, they often begin with 00 00 19 00 6f 08 00 00, or with 00 00 24 00 0b 00 0c 00.
They don't appear to begin with an Ethernet frame, an IP preamble, a UDP header, a TCP header, or any other expected data. When I search for my IPv4 MAC address within the data I often find it, but not always. Same with the IPv6 address. When I search for my IP address within the data, same deal.
Many of the packets seem to involve searching for or getting information about other Wifi networks.
Much of it seems to be operations to identify the (many) Wifi routers around me, but the format of that data is unknown to me.
Can anyone point me to a technical explanation of the payload bytes? Thanks.
I should also add that tcpdump itself has trouble reading these pcap files, which are generated on OS/X. The command to generate them is
/usr/sbin/tcpdump -s 0 -w output.pcap -vv -In -i en1
and the following command, which others recommended, does not properly dump them:
tcpdump -qns 0 -X -r output.pcap | less
In fact it generates lines that mention "tsft 1.0 Mb/s 2452 MHz 11g -78dB signal -91dB noise antenna 0 Beacon".
In fact it generates lines that mention "tsft 1.0 Mb/s 2452 MHz 11g -78dB signal -91dB noise antenna 0 Beacon".
What's wrong with that? It's dissecting radiotap headers in exactly the fashion it should.
You captured with the -I flag, meaning you captured in monitor mode. By default, on OS X (and, in most cases on Linux and *BSD), that captures with radiotap headers, giving radio-layer meta-data.
A pcap file has a "linktype" value in the file header; a pcap-ng file has a "linktype" value in each Interface Description Block for each network interface on which traffic was captured. Those values are described on the link-layer header types page of the tcpdump.org Web site. Your capture probably has a link-layer header type value of 127, which is LINKTYPE_IEEE802_11_RADIOTAP, and the packets begin with a radiotap header, followed by the 802.11 frame.
They absolutely always begin with 00 00.
That's the it_version field of the radiotap header, followed by the it_pad field; the only version of the radiotap header that currently exists is version 0, and the padding field is almost always if not always set to 0 (its value is irrelevant).
Looking at the bytes, they often begin with 00 00 19 00 6f 08 00 00, or with 00 00 24 00 0b 00 0c 00.
The 19 00 and 24 00 are the two-byte it_len field; it's little-endian, so 19 00 is 0x0019, or 25, and 24 00 is 0x0024, or 36. (19 00 is a bit suspicious, as it's not a multiple of 4.) That's the length of the radiotap header.