Modula-2 Reloaded

A Modern Typesafe & Literate Programming Notation

Site Menu

Project

Specification

Implementation

Recommendations

Reference

Needs Updating

Work in Progress

Wastebasket

Wiki Manual

edit SideBar

Serialised Scalar Format

WiP.SerialisedScalarFormat History

Hide minor edits - Show changes to output

2010-04-13 14:58 by benjk -
Changed lines 18-19 from:
* encoding of significands with up to 1000 digits and exponents with up to 18 digits
to:
* encoding of significands with up to 999 digits and exponents with up to 15 digits
Changed line 24 from:
* length: 2 octets (7..1023), total number of octets used to encode the scalar
to:
* length: 2 octets (8..1023), total number of octets used to encode the scalar
Changed lines 26-27 from:
* significand digit count: 2 octets (1..1000), number of digits encoded in significand field
* exponent digit count: 1 octet (0..18), number of digits encoded in exponent field
to:
* significand digit count: 2 octets (1..999), number of digits encoded in significand field
* exponent digit count: 1 octet (0..15), number of digits encoded in exponent field
Changed lines 71-72 from:
   digitB32 digitB32 ;    // allocated length, value between 7 and 1023
to:
   digitB32 digitB32 ;    // allocated length, value between 8 and 1023
Changed lines 77-78 from:
   digitB32 digitB32 ;    // digit count of significand, value between 1 and 1000
to:
   digitB32 digitB32 ;    // digit count of significand, value between 1 and 999
Changed line 80 from:
   digitB32 ;    // digit count of exponent, value between 0 and 18
to:
   digitB32 ;    // digit count of exponent, value between 0 and 15
2010-04-13 14:51 by benjk -
Added line 23:
* version: 1 octet, protocol version, at present always 1
Changed line 64 from:
   length encoding sigDigitCount expDigitCount
to:
    version length encoding sigDigitCount expDigitCount
Added lines 66-68:

version:
    digitB32 ; protocol version, at present the value is always 1
2010-04-10 12:39 by benjk -
Changed lines 25-26 from:
* significand digit count: 2 octets (1..1000), number of digits encoded for significand
* exponent digit count: 1 octet (0..18), number of digits encoded for exponent
to:
* significand digit count: 2 octets (1..1000), number of digits encoded in significand field
*
exponent digit count: 1 octet (0..18), number of digits encoded in exponent field
2010-04-10 11:41 by benjk -
2010-04-10 11:29 by benjk -
Changed line 110 from:
   in serialized. The output is shortened if the size of the passed in character array
to:
   in <serialized>. The output is shortened if the size of the passed in character array
Changed line 115 from:
   and passes the result back in value. *)
to:
   and passes the result back in <value>. *)
2010-04-10 11:28 by benjk -
Changed line 105 from:
The following conversion primitives should be implemented by all library defined scalar types
to:
The following conversion primitives should be implemented by all library defined scalar types to facilitate conversion between all numeric types even if no direct conversion path exists:
2010-04-10 11:27 by benjk -
Changed line 113 from:
PROCEDURE [FROM] fromSerializedScalar ( VAR value : <ScalarType>; serialized : ARRAY OF CHAR );
to:
PROCEDURE [FROM] fromSerialized ( VAR value : <ScalarType>; serialized : ARRAY OF CHAR );
2010-04-10 11:27 by benjk -
Changed lines 105-106 from:
The following conversion primitives
to:
The following conversion primitives should be implemented by all library defined scalar types
Changed line 108 from:
PROCEDURE toSerializedScalar ( value : <ScalarType>; VAR serialized : ARRAY OF CHAR );
to:
PROCEDURE [TO] toSerialized ( value : <ScalarType>; VAR serialized : ARRAY OF CHAR );
Changed line 113 from:
PROCEDURE fromSerializedScalar ( VAR value : <ScalarType>; serialized : ARRAY OF CHAR );
to:
PROCEDURE [FROM] fromSerializedScalar ( VAR value : <ScalarType>; serialized : ARRAY OF CHAR );
2010-04-10 11:23 by benjk -
Changed lines 110-111 from:
   in serialized. The output is shortened if the size of the passed in character array is
  insufficient to represent all available digits. *)
to:
   in serialized. The output is shortened if the size of the passed in character array
   is insufficient to represent all available digits. *)
2010-04-10 11:22 by benjk -
Changed lines 110-111 from:
   in serialized. The output is shortened if the size of the string passed in for serialized
 
  is insufficient to represent all available digits. *)
to:
   in serialized. The output is shortened if the size of the passed in character array is
  insufficient to represent all available digits. *)
2010-04-10 11:22 by benjk -
Changed lines 109-111 from:
(* Converts <value> to its serialised scalar representation and passes the result back in serialized. *)
to:
(* Converts <value> to its serialised scalar representation and passes the result back
   in serialized. The output is shortened if the size of the string passed in for serialized
  is insufficient to represent all available digits
. *)
2010-04-10 11:19 by benjk -
Changed lines 109-110 from:
(* Converts <value> to its serialised scalar representation
  and passes the result back in serialized. *)
to:
(* Converts <value> to its serialised scalar representation and passes the result back in serialized. *)
2010-04-10 11:15 by benjk -
Changed line 3 from:
The following describes an ASCII based serialisation format for numeric scalar values.
to:
The following describes an ASCII based serialisation format for numeric scalar values and conversion primitives to convert between numeric and serialised representation.
2010-04-10 11:12 by benjk -
Changed line 110 from:
   and passes the result back in serializedScalar. *)
to:
   and passes the result back in serialized. *)
2010-04-10 11:12 by benjk -
Changed line 108 from:
PROCEDURE toSerializedScalar ( value : <ScalarType>; VAR serializedScalar : ARRAY OF CHAR );
to:
PROCEDURE toSerializedScalar ( value : <ScalarType>; VAR serialized : ARRAY OF CHAR );
Changed line 112 from:
PROCEDURE fromSerializedScalar ( VAR value : <ScalarType>; serializedScalar : ARRAY OF CHAR );
to:
PROCEDURE fromSerializedScalar ( VAR value : <ScalarType>; serialized : ARRAY OF CHAR );
2010-04-10 11:10 by benjk -
Changed lines 108-112 from:
PROCEDURE toSerialisedScalar ( value : <ScalarType>; VAR serialised : ARRAY OF CHAR );
(* Converts value to its serialised scalar presentation and passes the result back in serialised. *)

PROCEDURE fromSerialisedScalar ( VAR value : <ScalarType>; serialised
: ARRAY OF CHAR );
(* Converts the passed in serialised scalar to a value of type <ScalarType> and passes the result back in value. *)
to:
PROCEDURE toSerializedScalar ( value : <ScalarType>; VAR serializedScalar : ARRAY OF CHAR );
(* Converts <value> to its serialised scalar representation
  and passes the result back in serializedScalar. *)

PROCEDURE fromSerializedScalar ( VAR value
: <ScalarType>; serializedScalar : ARRAY OF CHAR );
(* Converts the passed in serialised scalar to a value of type <ScalarType>
 
and passes the result back in value. *)
2010-04-10 11:07 by benjk -
Changed lines 101-113 from:
@]
to:
@]

!!!Conversion Primitives

The following conversion primitives

[@
PROCEDURE toSerialisedScalar ( value : <ScalarType>; VAR serialised : ARRAY OF CHAR );
(* Converts value to its serialised scalar presentation and passes the result back in serialised. *)

PROCEDURE fromSerialisedScalar ( VAR value : <ScalarType>; serialised : ARRAY OF CHAR );
(* Converts the passed in serialised scalar to a value of type <ScalarType> and passes the result back in value. *)

@]
2010-04-10 10:20 by benjk -
Changed lines 48-49 from:
Base10 serialisation is a radix 10 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0"), indicating 0, to decimal 57 ("9"), indicating 9.
to:
Base10 serialisation is a radix 10 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0") indicating 0, to decimal 57 ("9") indicating 9.
Changed lines 52-53 from:
Base16 serialisation is a radix 16 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0"), indicating 0, to decimal 63 ("?"), indicating 15.
to:
Base16 serialisation is a radix 16 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0") indicating 0, to decimal 63 ("?") indicating 15.
Changed line 56 from:
Base32 serialisation is a radix 32 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0"), indicating 0, to decimal 79 ("O"), indicating 31.
to:
Base32 serialisation is a radix 32 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0") indicating 0, to decimal 79 ("O") indicating 31.
2010-04-10 10:18 by benjk -
Changed lines 37-42 from:
* Numeric meta-data is always encoded in Base32 serialisation.
* If the value of the encoding field is "D", digit streams are encoded in Base10 serialisation.
* If the value of the encoding field is "H", digit streams are encoded in Base16 serialisation.

For all encodings, digits appear in highest to lowest significance.

to:
The digits in an encoded digit stream appear in highest to lowest significance order and represent the roots of a polynom of the form:
Added lines 41-44:
* Numeric meta-data is always encoded with radix 32, using Base32 serialisation.
* Significand and exponent digit streams are encoded as follows:
** radix 10, using Base10 serialisation if the value of the encoding field is "D"
** radix 16, using Base16 serialisation if the value of the encoding field is "H"
2010-04-10 10:12 by benjk -
Deleted line 44:
value = digit'_n_' * radix '^n^' + digit'_n-1_' * radix '^n-1^' ... + digit'_2_' * radix '^2^' + digit'_1_' * radix '^1^' + digit'_0_' * radix '^0^'
2010-04-10 10:11 by benjk -
Changed line 43 from:
value = digit'_0_' * radix '^n-0^' + digit'_1_' * radix '^n-1^' + ... + digit'_n-1_' * radix '^1^' + digit'_n_' * radix '^0^'
to:
value = digit'_0_' * radix '^n^' + digit'_1_' * radix '^n-1^' + digit'_2_' * radix '^n-2^' + ... + digit'_n-1_' * radix '^1^' + digit'_n_' * radix '^0^'
2010-04-10 10:10 by benjk -
Changed line 43 from:
value = digit'_0_' * radix '^n-0^ + digit'_1_' * radix '^n-1^ + ... + digit'_n-1_' * radix '^1^ + digit'_n_' * radix '^0^
to:
value = digit'_0_' * radix '^n-0^' + digit'_1_' * radix '^n-1^' + ... + digit'_n-1_' * radix '^1^' + digit'_n_' * radix '^0^'
2010-04-10 10:10 by benjk -
Added lines 42-43:

value = digit'_0_' * radix '^n-0^ + digit'_1_' * radix '^n-1^ + ... + digit'_n-1_' * radix '^1^ + digit'_n_' * radix '^0^
2010-04-10 10:05 by benjk -
Changed line 43 from:
encodedValue = digit'_n_' * radix '^n^' + digit'_n-1_' * radix '^n-1^' ... + digit'_1_' * radix '^1^' + digit'_0_' * radix '^0^'
to:
value = digit'_n_' * radix '^n^' + digit'_n-1_' * radix '^n-1^' ... + digit'_2_' * radix '^2^' + digit'_1_' * radix '^1^' + digit'_0_' * radix '^0^'
2010-04-10 10:04 by benjk -
Changed line 43 from:
encodedValue = digit'_n_' * radix '^n^' + digit'_n-1_' * radix '^n-1^' ... + digit'_0_' * radix '^0^'
to:
encodedValue = digit'_n_' * radix '^n^' + digit'_n-1_' * radix '^n-1^' ... + digit'_1_' * radix '^1^' + digit'_0_' * radix '^0^'
2010-04-10 10:03 by benjk -
Added lines 40-43:

For all encodings, digits appear in highest to lowest significance.

encodedValue = digit'_n_' * radix '^n^' + digit'_n-1_' * radix '^n-1^' ... + digit'_0_' * radix '^0^'
2010-04-10 09:59 by benjk -
Changed line 37 from:
* Numeric meta-data is encoded in a Base32 serialisation.
to:
* Numeric meta-data is always encoded in Base32 serialisation.
2010-04-10 09:58 by benjk -
Added lines 34-52:

!!!Encoding of Numeric Values

* Numeric meta-data is encoded in a Base32 serialisation.
* If the value of the encoding field is "D", digit streams are encoded in Base10 serialisation.
* If the value of the encoding field is "H", digit streams are encoded in Base16 serialisation.

!!!!Base10 Serialisation

Base10 serialisation is a radix 10 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0"), indicating 0, to decimal 57 ("9"), indicating 9.

!!!!Base16 Serialisation

Base16 serialisation is a radix 16 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0"), indicating 0, to decimal 63 ("?"), indicating 15.

!!!!Base32 Serialisation

Base32 serialisation is a radix 32 based ASCII encoding of numeric values. Digits are represented by the ASCII characters in the code point range from decimal 48 ("0"), indicating 0, to decimal 79 ("O"), indicating 31.

2010-04-10 09:46 by benjk -
Added line 32:
2010-04-10 09:46 by benjk -
Changed lines 32-33 from:

[1] If the value of the exponent's digit count field is zero, then no exponent is encoded
to:
[1] If the value of the exponent's digit count field is zero, then no exponent is encoded.
2010-04-10 09:45 by benjk -
Changed lines 20-22 from:
!!!Meta-Data

A
serialised scalar contains the following meta-data:
to:
!!!Data Fields

A
serialised scalar contains the following data fields:
Changed lines 29-30 from:
* exponent sign: 1 octet ("+" or "-"), sign of the exponent
* exponent digits: variable length, digit stream of exponent
to:
* exponent sign '^[1]^': 1 octet ("+" or "-"), sign of the exponent
* exponent digits'^[1]^': variable length, digit stream of exponent
Added lines 32-33:

[1] If the value of the exponent's digit count field is zero, then no exponent is encoded
2010-04-10 09:43 by benjk -
Added lines 19-31:

!!!Meta-Data

A serialised scalar contains the following meta-data:
* length: 2 octets (7..1023), total number of octets used to encode the scalar
* encoding: 1 octet ("D" or "H"), designator of radix system used to encode the scalar
* significand digit count: 2 octets (1..1000), number of digits encoded for significand
* exponent digit count: 1 octet (0..18), number of digits encoded for exponent
* significand sign: 1 octet ("+" or "-"), sign of the significand
* significand digits: variable length, digit stream of significand
* exponent sign: 1 octet ("+" or "-"), sign of the exponent
* exponent digits: variable length, digit stream of exponent
* terminator: 1 octet (ASCII NUL), string terminator
2010-04-10 09:32 by benjk -
Changed line 18 from:
* encoding of significands with up to 1000 digits and exponents of up to 18 digits
to:
* encoding of significands with up to 1000 digits and exponents with up to 18 digits
2010-04-10 09:31 by benjk -
Changed lines 18-20 from:
* encoding of significands with up to 1000 digits and a guaranteed minimum of 40 digits
* encoding of exponents with up to 30 digits and a guaranteed minimum of
18 digits
to:
* encoding of significands with up to 1000 digits and exponents of up to 18 digits
Changed lines 34-35 from:
   digitB32 digitB32 ;    // digit count of significand, value between 1 and 1018
to:
   digitB32 digitB32 ;    // digit count of significand, value between 1 and 1000
Changed line 37 from:
   digitB32 ;    // digit count of exponent, value between 0 and 31
to:
   digitB32 ;    // digit count of exponent, value between 0 and 18
2010-04-10 09:26 by benjk -
Added lines 18-19:
* encoding of significands with up to 1000 digits and a guaranteed minimum of 40 digits
* encoding of exponents with up to 30 digits and a guaranteed minimum of 18 digits
2010-04-10 09:17 by benjk -
Added line 8:
* avoid non-printable characters
2010-04-10 09:16 by benjk -
Changed lines 13-16 from:
* efficient calculation of the offset of the exponent's sign and digit stream
to:
* efficient calculation of the offset of the exponent's sign and digit stream
* encode all numeric meta-data in a power-of-two radix system
* encode digit streams using contiguous symbol sets for maximum efficiency
* avoid apostrophe, quotation marks, backquote and backslash in symbol sets
2010-04-10 09:09 by benjk -
Changed lines 1-2 from:
!!!Objective
to:
!!!Synopsis
Added lines 4-13:

!!!Objectives

The serialised format needs to meet the following objectives:
* store all meta-data fields at fixed offsets
* preserve the radix system of the value's internal representation
* store explicit length of the string holding the serialised value
* store explicit number of digits for both significand and exponent
* store the sign and digit stream of the significand at a fixed offset
* efficient calculation of the offset of the exponent's sign and digit stream
2010-04-10 08:56 by benjk -
Added lines 1-47:
!!!Objective

The following describes an ASCII based serialisation format for numeric scalar values.

!!!EBNF

[@
serialisedScalarFormat :
    length encoding sigDigitCount expDigitCount
    sigSign sigDigits ( expSign expDigits )? terminator ;

length :
    digitB32 digitB32 ;    // allocated length, value between 7 and 1023

encoding :
    "D" | "H" ;    // radix system, D for base 10, H for base 16

sigDigitCount :
    digitB32 digitB32 ;    // digit count of significand, value between 1 and 1018

expDigitCount :
    digitB32 ;    // digit count of exponent, value between 0 and 31

sigSign :
    "+" | "-" ;    // sign of significand

sigDigits :
    digitB10+ | digitB16+ ;    // digits of significand

expSign :
    "+" | "-" ;    // sign of exponent

expDigits :
    digitB10+ | digitB16+ ;    // digits of exponent

digitB10 :
    "0" .. "9" ;    // representing values between 0 and 9

digitB16 :
    "0" .. "?" ;    // representing values between 0 and 15

digitB32 :
    "0" .. "O" ;    // representing values between 0 and 31

terminator :
    ASCII(0) ;
@]