INI file

Basic INI files are a loosely defined de-facto standard; more info can be found here.

Basic properties

The Crawler INI files have the following properties:

• Section and entry names are case-insensitive by default (but Crawler has built-in support for case-sensitive INI files should the need arise).
• Comment lines are supported. Prefixing a line with a '#' or a ';' makes it a comment line. In-line comments are not supported: a single line is either a comment line or it is not - comments on lines with data are not supported. For example:

# This is a comment line
entry = test # test

means to set entry to "test # test". The trailing # test is not seen as a comment.

• Blank lines are allowed (and ignored)
• If an INI file is missing a section, then any entries without section are assumed to be in a default section [main]
• Duplicate entry names are allowed, and provide an 'override' mechanism. If an entry appears twice, the second appearance will 'win'.
• Entry values can be enclosed between double quotes (") in which case backslashes are used as an escape character as defined in JavaScript. If no double quotes are present, backslashes are not interpreted as escapes. When no double quotes are present, leading and trailing spaces are removed. The following entries are all equivalent:

data =     my data
data = "my data"
data=my data
data="my\x20data"
data="my\u0020data"

Enhancements

Crawler INI files have a few Crawler-specific enhancements.

Parent-child files

In a number of Crawler personalities, INI files are arranged in a parent-child relationship.

• It is possible to derive a new personality from an existing personality.
• Some personalities have some nested folder structures where INI files in the 'inner' folders use the INI files in the outer folders as parent files.

When two INI files have a parent-child relation, the child file 'inherits' all the contents of the parent's INI file. The child-INI can then either override certain entries in the parent INI (by repeating the same entry and section, and providing a different value), or perform string concatenation.

String concatenation

When an entry occurs multiple times in the same INI file or in a parent-child INI file arrangement, the use of '+=' is used to allow comma-separated string concatenation.

dataEntry = "some data"
...
dataEntry += "some more data, some more more data"

This will set the entry dataEntry to "some data, some more data, some more more data". Comma's are inserted between the concatenated values.

Auto-increment

INI files are not well suited for managing tabular, repetitive data.

The Crawler INI file format offers an enhancement to make repetitive (record-based) data entry easier. A line with just a ++ is interpreted as 'the first/next record follows'. The advantage is that it becomes easy to reorder complete data records in the INI file without needing to manually renumber individual entries.

The two following sections are equivalent:

[tableData]

name1=Kris
hours1=120

name2=John
hours2=112
extras2=12

name3=Will
hours3=99

[tableData]

++
name=Kris
hours=120

++
name=John
hours=112
extras=12

++
name=Will
hours=99

Conditional entries

Crawler-based INI files support conditional entries. This is done by means of a special, predefined section called '[conditionals]'.

In the conditionals section, there is a single predefined entry selectors. This entry is a list of comma-separated strings.

Each of these strings is called a selector. Their presence or non-presence drives the conditional entries.

Conditional entries have an entry name, followed by a question mark and a selector. These are only taken into account if the selector is present.

[conditionals]

selectors = xhtml, flow

[main]

personalityConfig= "./Personalities/default.ini"
personalityConfig?xhtml = "./Personalities/XHTML/config.ini"
personalityConfig?text = "./Personalities/Text/config.ini"
personalityConfig?hyperlinks = "./Personalities/Hyperlinks/config.ini"

This example sets the selectors entry to two separate selectors: xhtml and flow. The personalityConfig entry will then be set to "./Personalities/XHTML/config.ini" because the selectors text and hyperlinks are not set, so the entry with the selector xhtml 'wins'. It will override the initial 'default' entry for personalityConfig.