AGS4 - Rules

The following rules shall be used when creating an AGS4 data file (eg my_stuff.ags).

  • CSV Rule - means CSV file format rule
  • Data Rule - means and aata rule

Rule 1 - Character Set

CSV Rule

  • The data file shall be entirely composed of ASCII characters.

Warning

It is quite easy to violate this rule, eg Excel can insert a Windows character, or a cut paste from pdf.

See also

Rule 2 - GROUP’s required

CSV Rule

  • Each data file shall contain one or more data GROUP‘s.
  • Each data GROUP shall comprise a number of GROUP HEADER
  • rows must have one or more ref DATA rows

Rule 2a - CRLF New Line

  • Each row is located on a separate line, delimited by a new line

  • A new line consists of the two characters:

    • a carriage return (CR) (ASCII character 13)
    • followed by a line feed (LF) (ASCII character 10)

Note

Note

For Windows; CRLF is the default, and all text files are saved as CRLF by default.

Rule 2b - GROUP HEADER

Rule 3 - Data Descriptors

CSV Rule

  • Each row in the data file must start with a DATA DESCRIPTOR that defines the contents of that row.

  • The following Data Descriptors are used as described below:

    • Each GROUP row shall be preceded by the “GROUP” data descriptor
    • Each HEADING row shall be preceded by the “HEADING” data descriptor
    • Each UNIT row shall be preceded by the “UNIT” data descriptor
    • Each TYPE row shall be preceded by the “TYPE” data descriptor
    • Each DATA row shall be preceded by the “DATA” data descriptor

Example

"GROUP","HORN"
"HEADING","LOCA_ID","HORN_TOP","HORN_BASE","HORN_ORNT","HORN_INCL","HORN_REM","FILE_FSET"
"UNIT","","m","m","deg","deg","",""
"TYPE","ID","2DP","2DP","0DP","0DP","X","X"
"DATA","BH502","0.00","1.20","210","65","",""
"DATA","BH502","0.00","15.45","210","65","",""

Rule 4 - FIELD’s

CSV Rule

  • Within each GROUP, the DATA items are contained in data FIELDs. Each data FIELD contains a single data VARIABLE in each row. Each DATA row of a data file will contain one or more data FIELDs.
  • The GROUP row contains only one DATA item, the GROUP name, in addition to the Data Descriptor (Rule 3 - Data Descriptors).
  • All other rows in the GROUP have a number of DATA items defined by the HEADING row.

Rule 5 - Double Quotes

CSV Rule

  • DATA DESCRIPTORS, GROUP names, data field HEADING s, data field UNIT s, data field TYPE s, and data VARIABLE s shall be enclosed in double quotes (”...”).
  • Any quotes within a data item must be defined with a second quote, example below.
"he said ""hello"" to me"

Note

A double quote is ASCII char 34

Rule 6 - Comma Separated

CSV Rule

Note

A comma (,) is ASCII char 44

Rule 7 - Field Ordering

CSV Rule

Rule 8 - Units

Data Rule

  • Data VARIABLE s shall be presented in the units of measurement and type that are described by the appropriate data field UNIT and data field TYPE defined at the start of the GROUP within the GROUP HEADER rows.

Example

"GROUP","HORN"
"HEADING","LOCA_ID","HORN_TOP","HORN_BASE","HORN_ORNT","HORN_INCL","HORN_REM","FILE_FSET"
"UNIT","","m","m","deg","deg","",""
"TYPE","ID","2DP","2DP","0DP","0DP","X","X"

Rule 9 - Data Dictionary

Data Rule

Rule 10 - Validation

Data Rule

  • HEADING s are defined as KEY, REQUIRED or OTHER.
    • KEY fields are necessary to uniquely define the data.
    • REQUIRED fields are necessary to allow interpretation of the data file.
    • OTHER fields are included depending on the scope of the data file and availability of data to be included.

Rule 10a - KEY

  • In every GROUP, certain HEADING s are defined as KEY.
  • There shall not be more than one row of data in each GROUP with the same combination of KEY field entries.
  • KEY fields must appear in each GROUP, but may contain null data (see Rule 12 - Blank/Null Data).

Rule 10b - REQUIRED

  • Some HEADING s are marked as REQUIRED.
  • REQUIRED fields must appear in the data GROUP s where they are indicated in the AGS4 Data Dictionary.
  • These fields require data entry and cannot be null (i.e. left blank or empty).

Rule 10c - PARENT GROUP

  • Links are made between data rows in GROUP s by the KEY fields.
  • Every entry made in the KEY fields in any GROUP must have an equivalent entry in its PARENT GROUP.
  • The PARENT GROUP must be included within the data file.
  • GROUP parentage is defined in Section 7.3.

See also

Rule 12 - Blank/Null Data

CSV Rule

  • Data does not have to be included against each HEADING unless REQUIRED.
  • The data FIELD can be null
  • a null entry is defined as “” (two quotes together, no space etc).

Example

"GROUP","HORN"
"HEADING","LOCA_ID","HORN_TOP","HORN_BASE","HORN_ORNT","HORN_INCL","HORN_REM","FILE_FSET"
"UNIT","","m","m","deg","deg","",""
"TYPE","ID","2DP","2DP","0DP","0DP","X","X"

Rule 13 - Mandatory Project Details

Data Rule

  • Each data file shall contain the PROJ GROUP which shall contain only one data row:
  • As a minimum, shall contain data under the headings defined as REQUIRED (Rule 10b - REQUIRED).
"GROUP","PROJ"
"HEADING","PROJ_ID","PROJ_NAME","PROJ_LOC","PROJ_CLNT","PROJ_CONT","PROJ_ENG","PROJ_MEMO","FILE_FSET"
"UNIT","","","","","","","",""
"TYPE","ID","X","X","X","X","X","X","X"
"DATA","7845","Trumpington Sewerage","Trumpington","Trumpington District Council","GeoI Ltd","Geo-Knowlege","Example AGS file - assocaited files are not included","FS001"

Rule 14 - TRAN GROUP

Data Rule

Rule 15 - UNIT GROUP

Data Rule

  • Each data file shall contain the UNIT GROUP to list all units used within the data file.
  • Every unit of measurement entered in the UNIT row of a GROUP or data entered in a FIELD where the field TYPE is defined “PU” (for example ERES_RUNI, GCHM_UNIT or MOND_UNIT FIELDs) shall be listed and defined in the UNIT GROUP.

Example snippet

"GROUP","UNIT"
"HEADING","UNIT_UNIT","UNIT_DESC","FILE_FSET"
"UNIT","","",""
"TYPE","X","X","X"
"DATA","m","metre",""
"DATA","mm","millimetre",""
"DATA","yyyy-mm-dd","year month day",""
"DATA","yyyy-mm-ddThh:mm","year month day hours minutes",""
"DATA","hh:mm:ss","hours minutes seconds",""
"DATA","%","percentage",""
"DATA","g/l","grams per litre",""
"DATA","mg/l","milligrams per litre",""
"DATA","%vol","percentage volume",""
"DATA","deg","degrees",""
"DATA","bar","bar",""
"DATA","-","No Units",""
"DATA","degC","degree Celsius",""
"DATA","l/min","litres per minute",""

Rule 16 - Abbreviations

Data Rule

  • Each data file shall contain the ABBR GROUP when abbreviations have been included in the data file.
  • The abbreviations listed in the ABBR GROUP shall include definitions for all abbreviations entered in a FIELD where the data TYPE is defined as “PA” or any abbreviation needing definition used within any other heading data type.

Rule 16a

  • Where multiple abbreviations are required to fully codify a FIELD
  • The abbreviations shall be separated by a defined concatenation character.
  • This single concatenation character shall be defined in TRAN_RCON. The default being “+” (ASCII character 43)
  • Each abbreviation used in such combinations shall be listed separately in the ABBR GROUP.
  • e.g. “CP+RC” must have entries for both “CP” and “RC” in ABBR GROUP, together with their full definition.

Rule 17 - TYPE

Data Rule

  • Each data file shall contain the TYPE GROUP to define the field TYPE s used within the data file.
  • Every data type entered in the TYPE row of a GROUP shall be listed and defined in the TYPE GROUP.

Rule 18 - Dictionary

Data Rule

  • Each data file shall contain the DICT GROUP where non-standard GROUP and HEADING names have been included in the data file.

Rule 18a

  • The order in which the user-defined HEADING s are listed in the DICT GROUP shall define the order in which these HEADINGS are appended to an existing GROUP or appear in a user-defined GROUP.
  • This order also defines the sequence in which such HEADINGS are used in a heading of data TYPE Record Link (Rule 11 - Record Links).

Rule 19 - Group headings

Data Rule

  • A GROUP name shall:
    • not be more than four characters long
    • and shall consist of uppercase letters and numbers only.
# Good
"GROUP","TBAA"
"GROUP","TB01"

# Bad
"GROUP","BAD"
"GROUP","abc"
"GROUP","TB_1"
"GROUP","TX 3"

Rule 19a - Heading length

  • A HEADING name shall:
    • not be more than nine characters long
    • and shall consist of uppercase letters, numbers or the underscore character only.

Rule 19b - Heading Prefix

  • HEADING names shall start with the GROUP name followed by an underscore character.
    • e.g. "NGRP_HED1"

  • Where a HEADING refers to an existing HEADING within another GROUP, the HEADING name added to the group shall bear the same name. - e.g. "SAMP_REF" in the "SAMP" GROUP - or "CMPG_TESN" in the "CMPT" GROUP.

Rule 20 - Additional files

Data Rule

  • Additional computer files (e.g. digital images) can be included within a data submission.
  • Each such file shall be defined in a FILE GROUP.
  • The additional files shall be transferred in a sub-folder named FILE.
  • This FILE sub-folder shall contain additional sub-folders each named by the FILE_FSET reference.
  • Each FILE_FSET named folder will contain the files listed in the FILE GROUP.