LDBS disc image format v0.5

John Elliott, 25 November 2020

Introduction

LDBS is a disc image format originally designed for internal use in LibDsk, with the intended use case being for archiving FM / MFM floppy discs. This specification is provided for comment and suggestions, in case the format proves useful in other situations.

The suggested file extension is .ldbs.

The reference implementation for this version of LDBS is ldbs.c / ldbs.h, supplied with LibDsk v1.5.14.

The file structure is defined at two levels: a generic block store, and the specific block types used to specify a disc image.

Rationale: Why a new disc image format? As mentioned above, it was designed for use as an intermediate format in LibDsk. The aim is to have feature parity with CPCEMU extended .DSK, but in a more extensible format with greater support for in-place rewrites. It is not, for example, possible to reformat a track from 8 to 9 512-byte sectors in an EDSK file, without rewriting all subsequent tracks.

Change history

0.5 (25 November 2020)

Added a textual version of the format, and utilities ldbs2txt and txt2ldbs to convert to / from it.

0.4 (9 July 2019)

Added support for sectors whose size is not a power of 2.
Added support for Macintosh GCR encoding.
Added support for 'MBIN' and 'RSRC' blocks (MacBinary header and Apple resource fork).

0.3 (8 March 2017)

Altered the definition of the track and sector headers to support some CPC copy-protection methods and add further extensibility. Since this is a breaking change to the binary file format, I have increased the version number in the header.

0.2 (3 March 2017)

Added a warning against storing file offsets in custom blocks.
Implemented a few more helper methods. The disc image format itself remains unchanged.

0.1 (16 March 2016)

Original version.

Data storage: Block Store

All 16-bit and 32-bit values are stored in Intel format (low byte first).

File offsets and block lengths are 32 bit little-endian doublewords. To avoid signed / unsigned issues, they should not exceed 2^31.

Header

All LDBS files begin with a 20-byte header:

Offset	Type	Meaning
0x0000	4 bytes	LDBS magic number: `0x4C 0x42 0x53 0x01` ('`LBS\01`')
0x0004	4 bytes	File type. For a disc image this is `0x44 0x53 0x4B 0x02` ('`DSK\02`'). If it is `0x44 0x53 0x4B 0x01` ('`DSK\01`') then it is in LDBS 0.2 or earlier format. Such disc images are not likely to be in wide circulation and I don't think you need to bother implementing support for them. The supplied `ldbs_v2` utility can convert disc images in the earlier format to the current version.
0x0008	offset	Offset of first block in 'used' linked list. Zero means there are no used data blocks.
0x000C	offset	Offset of first block in 'free' linked list. Zero means there are no free data blocks.
0x0010	offset	Offset of track directory. Zero means there is no track directory.

If the offsets at 0x0008 / 0x000C / 0x0010 are non-zero, they must be file offsets of valid data blocks. It is an error for them to point to any other location in the file.

Within the block store layer the track directory is optional, but a valid LDBS disc image must have a track directory.

Although the block store layer is currently only defined as containing a disc image, it would be usable as a container for other file formats; this would be indicated by a different value at offset 0x0004. The use of the block store for other file formats is not covered further by this document.

Data blocks

The header is followed by one or more data blocks (a file with no data blocks is structurally valid, but would not contain a track directory). There is no constraint on the order of blocks in a file; a track header does not have to be stored together with its associated sectors, for example.

For an LDBS disc image, is recommended that a given data block should not exceed 64k in size.

There are no alignment requirements; blocks can begin at any file offset.

Each data block has its own 20-byte header:

Offset	Type	Meaning
0x0000	4 bytes	Block signature: `0x4C 0x44 0x42 0x01` ('`LDB\01`')
0x0004	4 bytes	Block type. 4-byte value as defined below. `0x00 0x00 0x00 0x00` indicates a free block, other values indicate a used block.
0x0008	length	Length of block on disc (not including this 20-byte header)
0x000C	length	Length of block contents, less than or equal to length of block on disc. Should be set to zero for a free block.
0x0010	offset	Offset of next block in used / free linked list. Zero if this is the last block in the list.

The header is followed immediately by the block data.

If a block is recorded in the track directory (see below) its block type is used to identify it, and so must be unique within the file. The block type is not required to be unique for blocks not recorded in the directory.

If the offset at 0x0010 is non-zero, it must point to another valid data block.

If the length of the block on disc is greater than the length of the block contents, an implementation is encouraged to fill the remainder with zero bytes, but is not obliged to. Similarly a free block should be filled with zeroes, but is not required to be.

Data blocks can be enumerated either by stepping through the file in linear order, or by following the used / free linked lists.

It is an error for there to be a 'free' block on the 'used' linked list, or a 'used' block on the 'free' linked list.

Operations

Typical operations performed at block level are:

Get the length of a block. Given an offset, load the block header at that offset and return its length.
Load a block. Given an offset, load the block header and data at that offset.
Add a block. Search the free list for an entry greater than or equal to the required size. If one is found, remove the entry from the free list, add it to the used list, and populate with the data. If not found, append a new block of the required size to the end of the file, and add it to the used list.
Delete a block. Remove the block from the used list, add it to the free list, and zero its block type.
Update a block. If the new size is less than or equal to the block's current size on disc, write the data into the block at its current location. Otherwise, treat as delete followed by add.

Note that there is no requirement to coalesce adjacent free blocks, or to split a block if the size requested is less than its current size on disc.

Data storage: Disc image

Track directory

A disc image must contain a track directory block. The intention is that an implementation will use the track directory to locate data rather than performing linear searches of the file or walking its linked lists.

The format of the track directory is:

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x44 0x49 0x52 0x01` ('`DIR\01`')
0x0014	16-bit word	Number of directory entries
0x0016	8 bytes per entry	Directory entries

The format of a directory entry is one of Txxx, INFO, CREA, DPB, GEOM, MBIN, RSRC, or a custom block type (beginning with a lowercase 'a'-'z'). In each case, this is the block type of the corresponding block in the store.

Offset	Type	Meaning
0x0000	1 byte	0x54 ('T')
0x0001	2 bytes	Cylinder number
0x0003	1 byte	Head number
0x0004	offset	Offset of track header block

Reference to a track header. There is one of these for each track in the disc image.

Offset	Type	Meaning
0x0000	4 bytes	'INFO'
0x0004	offset	Offset of comment block

Reference to a comment block. There is at most one of these in the disc image file.

Offset	Type	Meaning
0x0000	4 bytes	'CREA'
0x0004	offset	Offset of creator block

Reference to a creator block. There is at most one of these in the disc image file.

Offset	Type	Meaning
0x0000	4 bytes	'GEOM'
0x0004	offset	Offset of geometry block

Reference to a geometry block. There is at most one of these in the disc image file.

Offset	Type	Meaning
0x0000	4 bytes	'DPB '
0x0004	offset	Offset of DPB block

Reference to a CP/M Disk Parameter Block block. There is at most one of these in the disc image file. It will only be present on discs containing a CP/M filesystem.

Offset	Type	Meaning
0x0000	4 bytes	'MBIN'
0x0004	offset	Offset of MacBinary header block

Reference to a MacBinary header block. There is at most one of these in the disc image file.

Offset	Type	Meaning
0x0000	4 bytes	'RSRC'
0x0004	offset	Offset of Macintosh resource block

Reference to a Macintosh resource fork. There is at most one of these in the disc image file.

Custom block types

Other block types than the ones listed here may be defined in later versions of this spec; implementations should ignore unknown block types. It may, for example, be useful at some point to create a 'BOOT' block type containing the keystrokes necessary to boot a disc, or a 'PICT' block containing a picture of the disc.

All future block types defined by this spec will start with a capital letter, 'A'-'Z'. Block types beginning with a lowercase letter 'a'-'z' can be used for private purposes — there is, of course, no guarantee that someone else won't pick the same block type for their own use.

You should avoid storing file offsets in your custom blocks. There is nothing stopping an LDBS implementation from rearranging blocks in a file so they end up at different offsets (for example, to remove unused space). However, such an implementation won't be aware of offsets held in custom blocks, and so won't update them. This isn't just a theoretical danger: ldbs_clone() reverses the order of blocks when duplicating an LDBS file.

The LibDsk TeleDisk, CopyQM and QRST drivers use custom blocks to hold details from the original file headers which aren't used by other disc image formats. These are, respectively, 'utd0', 'ucqm', and 'uqrs'.

Track header

Offset Type Name Meaning

0x0000 20 bytes LDBS block header. Type field contains 0x54 cyl cyl head, corresponding to directory entry.

0x0014 2 bytes [LDBS 0.3+] Length of fixed track header in bytes (= offset of first sector header less 0x0014). Currently 0x000C, but you should not rely on this; future extensions may require extra bytes to be added to the track header.

0x0016 2 bytes [LDBS 0.3+] Length of each sector descriptor in bytes. Currently 0x0012, but you should not rely on this; future extensions may require extra bytes to be added to the sector header.

0x0018 2 bytes count Number of sector entries in the track header.

0x001A

1 byte

datarate

Data rate when track was recorded:

0 => unknown
1 => Single density (125 kbit/s or 150 kbit/s FM) or double density (250 kbit/s or 300 kbit/s MFM)
        Also used for Macintosh GCR.
2 => High density (500 kbit/s MFM, or [unlikely] 250 kbit/s FM)
3 => Extended density (1000 kbit/s MFM, or [unlikely] 500 kbit/s FM)

Note: The 250 kbit/s and 300 kbit/s rates (and their FM equivalents) are combined because the same disc may appear to use either rate, depending whether it is read in a 360k drive or a 1.2Mb drive.

0x001B

1 byte

recmode

Recording mode used for the track:

Value	Meaning
0x00	unknown
0x01	FM
0x02	MFM
0x10-0x2F	[LDBS 0.4+] Macintosh GCR. The value is defined as the GCR format byte (byte 0x51 of a Disk Copy 4.2 file) masked with 0x1F, plus 0x10. This leads to the values: 0x12 => Macintosh 0x14 => Apple II ProDos 0x22 => Lisa

0x001C 1 byte gap3 Format gap length

0x001D 1 byte filler Default filler byte

0x001E 2 bytes total_len [LDBS 0.3+] Approximate length of track (including address marks and gaps) in bytes. For some timing-based copy protection schemes. Zero if this value was not captured when the disc was imaged.

0x0020 [LDBS 0.3+] Future versions may add extra bytes here. See offset 0x0014

See offset 0x0014 See offset 0x0016 sector [LDBS 0.3+] Sector headers

Each sector descriptor is formed:

Offset	Type	Name	Meaning
0x0000	1 byte	`id_cyl`	Sector ID: Cylinder
0x0001	1 byte	`id_head`	Sector ID: Head
0x0002	1 byte	`id_sec`	Sector ID: Sector
0x0003	1 byte	`id_psh`	Sector ID: Size (0 => 128, 1 => 256, 2 => 512 ... 7 => 8192) Note that the actual sector size may not be a power of 2; see `datalen` at offset 0x0010 for more details.
0x0004	1 byte	`st1`	8272 status 1. The following bits may be set if the archiver encountered errors reading this sector: Bit 7: End of cylinder Bit 5: Data error in ID or data field Bit 2: No data Bit 0: Missing address mark in ID or data field
0x0005	1 byte	`st2`	8272 status 2. The following bits may be set if the archiver encountered errors reading this sector, or if the sector has a deleted data control mark: Bit 6: Control mark (sector marked as deleted data) Bit 5: Data error in data field Bit 0: Missing address mark in data field
0x0006	1 byte	`copies`	Number of copies held. Some copy-protection systems use a 'weak' sector which returns different values each time it is read. This is simulated by holding multiple copies of the sector, and returning a random one each time it is read. If this is zero, the sector is treated as blank (ie, entirely filled with the `filler` byte at 0x0007).
0x0007	1 byte	`filler`	Filler byte if sector is blank
0x0008	offset	`blockid`	Offset of sector data block for this sector. If number of copies is zero, this must be zero too.
0x000C	2 bytes	`trail`	Number of trailing bytes. If nonzero, then the specified number of CRC and gap bytes follow each copy of the sector itself. If no copies of the sector are held, the number of trailing bytes must be 0. For Macintosh GCR discs, each sector is expected to have 12 trailing bytes containing sector metadata.
0x000E	2 bytes	`offset`	Approximate offset of sector within track. This is used for some timing-based copy protection schemes. If this value was not captured it will be zero for all sectors on the track.
0x0010	2 bytes	`datalen`	[LDBS 0.4+] Length of sector data in bytes (not including any trailing bytes). Files written by LDBS 0.3 omit this field; this can be determined from the sector descriptor length at offset 0x0016 of the track header. If this field is not present (or is present and zero), sector data length must be deduced from the `id_psh` field, and thus can only be a power of two.
0x0012		`id_cyl`	[LDBS 0.3+] Future versions may add extra bytes here. See offset 0x0016 of track header.

Sector data

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x53 cylinder head sector`. Note that `cylinder` and `head` are the sector's actual physical location on the disc, which may not be the same as the values in its ID header.
0x0014	As specified in descriptor	The sector data. In normal circumstances the length should match the descriptor in the track header: (data bytes + trailing bytes) * number of copies. However implementations should be prepared to handle the data being either shorter or longer than the expected size; this may arise if the LDBS file was converted from a disc image file with the same anomaly, or an LDBS 0.3 file containing sectors whose size is not a power of two.

Since the track directory does not record the locations of individual sectors, the block type field is not required to be unique. If an LDBS file contained more than 256 cylinders, for example, sectors on cylinder 256 would have the same block type as those on cylinder 0. Similarly if a track contained several copies of the same sector, all these copies would have the same block type.

Comment block

The comment block is optional, and contains a human-readable comment describing the disc image.

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x49 0x4E 0x46 0x4F` ('INFO')
0x0014	As specified in header	Disk comment. Lines should be separated with DOS-style newlines ('\r\n').

On character sets: If possible, stick to ASCII. Failing that, UTF-8.

Creator block

The creator block is optional, and contains human-readable text naming the utility that created the disc image.

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x43 0x52 0x45 0x41` ('CREA')
0x0014	As specified in header	Creator, ASCII, no newlines.

DPB block

The DPB block is optional, and contains a CP/M Plus Disk Parameter Block describing the filesystem.

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x44 0x50 0x42 0x20` ('DPB ')
0x0014	17 bytes	CP/M DPB

Rationale: CP/M filesystems are not self-describing. If the disc image was generated by an emulator or utility that is aware of the CP/M filesystem, it may be helpful to record the filesystem parameters used. This is also necessary for round-trip compatibility with the YAZE YDSK format.

Geometry block

The geometry block is optional. It contains the drive geometry used by LibDsk when it last formatted a track in the disc image. On a 'straightforward' disc image where all tracks have the same format, this will give the geometry of the entire file. On discs with varying numbers of sectors per track (such as Macintosh GCR) it will inevitably fall short.

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x47 0x45 0x4F 0x4D` ('GEOM')
0x0014	Byte	Sidedness: the order in which to process tracks. 0 => Single sided, or alternating sides. Track `n` is cylinder (`n` / `heads`) head (`n` % `heads`). 1 => Out and back. Tracks go (head 0) 0,1,2,3,...,37,38,39 then (head 1) 39,38,37,...2,1,0 2 => Out and out. Tracks go (head 0) 0,1,2,3,...,37,38,39 then (head 1) 0,1,2,3,...,37,38,39 3 => Extended surface. As SIDES_ALT, but sectors on head 1 identify as head 0, with numbers in sequence eg: Head 0 has sectors 1-9, head 1 has 10-18
0x0015	Word	Number of cylinders
0x0017	Byte	Number of heads
0x0018	Byte	Number of sectors per track
0x0019	Byte	First physical sector number
0x001A	Word	Bytes per sector
0x001C	Byte	Data rate: 0 => 500 kbit/s, 1 => 300 kbit/s, 2 => 250 kbit/s, 3 => 1 mbit/s
0x001D	Byte	Read/write gap
0x001E	Byte	Format gap
0x001F	Byte	Recording mode: 0 => MFM, 1=> FM, 0x10-0x2F => Macintosh GCR
0x0020	Byte	Complement flag, 1 if bytes are stored complemented
0x0021	Byte	Disable multitrack read/writes
0x0022	Byte	Do not skip deleted data

Note that the representation of the data rate and MFM recording mode isn't the same as in a track header. This is for binary compatibility with LibDsk. FM and GCR recording modes have the same values in both structures.

Rationale: If the imaged disc contains no superblock or other means to specify its own format, this can be used to suggest a geometry for accessing it.

Macintosh data

To preserve data from files created under classic MacOS (for example by DiskCopy 4.2), two blocks are defined to hold the Finder information and resource fork of a file:

MBIN block

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x4D 0x42 0x49 0x4E` ('MBIN')
0x0014	128 bytes	Finder information, in the form of a MacBinary file header. Some fields (such as the length of the data fork) will be irrelevant; the type and creator code may be more to the purpose.

RSRC block

Offset	Type	Meaning
0x0000	20 bytes	LDBS block header. Type field contains `0x52 0x53 0x52 0x43` ('RSRC')
0x0014	Arbitrary length	Macintosh resource fork

When converting from an LDBS disc image to a Macintosh file, caution is advised when using the MBIN / RSRC blocks. It's entirely possible that since the file was originally created, it's been changed sufficiently that the blocks are out of sync. For example, Disk Copy 4.2 stores copies of the disc checksums in the resource fork of disc images it creates; if the contents of the disc image have changed, the checksums will need to be recalculated when the new resource fork is written out.

Compression

The only form of compression is omitting sectors that are blank or unreadable (by setting the 'copies' field in the sector header to 0). Should it prove necessary to compress LDBS disc images further, this should be done using an external utility such as gzip (LibDsk can transparently decompress gzipped disc images).

LDBS as text

The utilities ldbs2txt and txt2ldbs convert LDBS disc images to / from a text format that is hopefully human-readable and even human-editable. The suggested file extension is .LDBST.

The text format is similar to a Windows .INI file, with each section introduced by a [Section] heading in square brackets, followed by Key = Value lines.

Comments are preceded by the semicolon ; or hash # characters, unless these are within a string (see below).

Except within strings, all text is case-insensitive. Numeric values can be decimal, or hex (preceded by 0x).

The Type = and Data = lines can be followed either by a string, or by a hex dump. A string is denoted by quote marks (for example: Type = "utd0"). The following two-character sequences have special meaning within a string:

\n	Newline
\t	Tab
\r	Carriage return
\"	Literal quote
\\	Literal backslash

A hex dump is indicated by { and continues until }. It is allowed to cover multiple lines. Within the brackets, only hexadecimal digits matter; other characters are ignored. For example, an 11-byte hex sequence could be stored as:

Data = {54 44 00 00 32 15 00 01  
	00 00 01
        }

[LDBS] section

The first line of the file should be the single word [LDBS] (preceded by a UTF-8 BOM if appropriate). This allows its format to be detected by utilities that look for a signature at a fixed offset.

There are currently no variable / value pairs in the [LDBS] section; if present, they would correspond to general settings of the disk image.

The sections following [LDBS] can come in any order, except that all the sectors in a track must follow its [Track] section.

[Track] section

Each [Track] section corresponds to a Txxx track header block in a binary LDBS file. It must contain at least the following entries:

Cylinder =: Location of the track on the disk, numeric
Head =: Which side of the disk the track is on, numeric

It should also contain these fields, corresponding to the members of an LDBS track header:

DataRate =: Data rate: Unknown, SD, HD or ED
RecMode =: Recording mode: Unknown, FM, MFM, GCR_Mac, GCR_Lisa, GCR_Prodos or GCR_Mac_n where 0 ≤ n ≤ 15
GAP3 =: Format gap length, numeric
Filler =: Default format filler byte, numeric
TotalLength =: Approximate length of track, numeric (if omitted, assumed to be zero)

[Sector] section

All the sectors in a track should follow the [Track] section for that track. The entries correspond to those in the LDBS sector descriptor:

ID.Cylinder =: Sector ID: Cylinder, numeric
ID.Head =: Sector ID: Head, numeric
ID.Sector =: Sector ID: Sector, numeric
ID.PSH =: Sector ID: Sector size (0 => 128, 1 => 256, 2 => 512 etc.) Numeric. For actual sector size see DataLen.
Status1 =: 8272 status 1, numeric.
Status2 =: 8272 status 2, numeric.
Copies =: Number of copies held, numeric.
Filler =: Filler byte used if sector blank, numeric.
DataLen =: Length of sector data in bytes, omitting any trailing bytes. Numeric.
TrailBytes =: Number of CRC / gap bytes following each copy of the sector data. If omitted assumed to be zero. Numeric.
Offset =: Approximate offset of sector within track. If omitted, assumed to be zero. Numeric.
Data =: The sector data, as a hex dump. Only present if Copies is nonzero.

[Creator] section

Corresponding to the LDBS CREA creator info block, this section contains a single Data= line (either string or hex dump) naming the utility that created the LDBS disc image.

[Comment] section

Corresponding to the LDBS INFO comment block, this section contains a single Data= line (either string or hex dump) holding the LDBS comment field.

[DPB] section

This corresponds to the LDBS DPB (CP/M DPB) block. It contains up to 12 entries, all numeric:

SPT, BSH, BLM, EXM, DSM, DRM, AL0, AL1, CKS, OFF, PSH, PHM

[Geometry] section

This corresponds to the LDBS GEOM (LibDsk geometry) block. It contains up to 13 entries:

Sidedness =: One of Alt, OutBack, OutOut, ExtSurface
Cylinders =: Count of cylinders. Numeric.
Heads =: Count of heads. Numeric.
Sectors =: Number of sectors per track. Numeric.
SecBase =: First physical sector number. Numeric.
SecSize =: Bytes per sector. Numeric.
DataRate =: Data rate. One of SD, DD, HD or ED.
RWGap =: Read/write gap. Numeric.
FmtGap =: Formatting gap. Numeric.
RecMode =: Recording mode. One of FM, MFM, GCR_Mac, GCR_Lisa, GCR_Prodos or GCR_Mac_n where 0 ≤ n ≤ 15
Complement =: Bytes stored complemented? (Y or N).
MultiTrack =: Multitrack reads/writes? (Y or N).
SkipDeleted =: Skip deleted data? (Y or N).

[Block] section

This corresponds to all other types of LDBS block that may be found in an LDBS file (eg: MBIN, RSRC or user-defined blocks). It contains two lines, in the following order:

Type= : The block type, either as "string" or { hex }.
Data= : The contents of the block, again either as "string" or { hex }.

Reference Implementation

ldbs.h / ldbs.c in LibDsk 1.5.14 contain an X11-licensed reference implementation of LDBS. A few sample utilities are also supplied:

dsk2ldbs: Converts DSK / EDSK disc images to LDBS.
ldbs2dsk: Converts LDBS disc images to DSK / EDSK.
ldbs2raw: Converts LDBS disc images to a raw data dump.
ldbs2txt: Converts an LDBS disc image to a text file.
txt2ldbs: Converts a text LDBS image back to binary.
ldbstest: Sanity test of block layer functions.
ldbsdump: Displays the contents of an LDBS file.
ldbscopy: Copies one LDBS file to another using the ldbs_clone() function. As currently written this happens to reverse the order of blocks in the file, so all block offsets will change.
ldbs_v2: Upgrades a file in LDBS v0.1 or v0.2 format to the current specification.