Table of Contents

  1. CSV Header Meta Details

A CSV Feed will bring in a tabular dataset from a target CSV source. The feed will be designated as a “CSV” feed in the data pipeline (example below).

In addition to the common properties (as determined by the connection type), the following properties are used when configuring a CSV Feed:

Property Name
Description
Field Delimiter This value defines what character is used to separate columns of data. The default value is a comma.
Header Meta Delimiter This allows users to configure what character is used to separate out meta data in the CSV’s header. Header Meta is typically used to handle typing in CSV data so that values can be identified as NUMBERs or DATEs instead of standard STRINGs.
Poll Interval (seconds)
This poll rate determines how often the server will check the CSV data for updates.
Enable Server Subscription If set, the server will subscribe to the feed, just as a client widget would. This means that the data and any resources that would otherwise be allocated “on-demand” for the first user to view a Visualization that leverages the data produced by this feed are allocated when the server is started and maintained as long as this feed is configured.

CSV Header Meta Details #

Header meta data is optional, and typically CSVs do not include this information. However, for convenience it can be added either manually (in the case with CSV files on the filesystem) or via external scripts with Shell Exec feeds. This can save the step of casting the fields and setting which fields are primary keys once loaded into edgeData – otherwise all fields are read in as STRINGs without any primary key(s).

An example header with meta information and first row of data is shown below:

Name|STRING|PK,Description|STRING,Total|NUMBER,LastUpdate|DATE|yyyy-MM-dd
ExampleRow,Description Here,123,2013-06-24

The following meta attributes are allowed:

Meta Attribute
Description
STRING The field is of type STRING
NUMBER The field is of type NUMBER
BOOLEAN The field is of type BOOLEAN
DATE

The field is of type DATE. This attribute requires a second attribute to specify the date format:

  • seconds: for UNIX epoch time in seconds – e.g: 1486102805
  • millis: for UNIX epoch time in milliseconds – e.g: 1486102805000
  • custom date format: specify a date format pattern to parse the date using the syntax as defined in SimpleDateFormat. See the example block above.

    Uppercase vs. lowercase is important when working with a custom date format string.  Example: MM/dd/yyyy.
    Java distinguishes between y and Y. See https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html.
PK The field is a primary key. Multiple PKs may be defined.