Live File Processor Agent Example – E-BAM

<< Click to Display Table of Contents >>

Navigation:  Live > Agents > File Agents > File Processor > Examples >

Live File Processor Agent Example – E-BAM

1.Add the following settings to the ST_CONFIG table (CONFIG_SECTION = 'Live.FileProcessor', CONFIG_KEY = 'Agent').

 

OBJECT_TYPE

OBJECT_VALUE

STRING_VALUE

Description

E-BAM

folderWatcherPath

C:\EarthSoft\Live\E-BAM


E-BAM

folderWatcherFilterRegex

^(.*[^\s].*)_
([^_]*[^\s_][^_]*)_
([^_]*[^\s_][^_]*)_
([^_]*)$

Ignores files whose names do not match this pattern,

e.g., VNTR_AM_MV-1_PM10_06162019.csv

 

DT_LOGGER.FACILITY_ID = DT_FACILITY.FACILITY_ID where FACILITY_CODE = 'VNTR_AM' (facility codes may contain an '_').

DT_LOGGER.SYS_LOC_CODE = 'MV-1'.

DT_LOGGER_SERIES.LOCATION_PARAM_CODE = 'PM10'.

E-BAM

facilityCodeFileNameRegexGroups

^(?'a'.*)_(.*)_(.*)_(.*)$

DT_LOGGER.FACILITY_ID = DT_FACILITY.FACILITY_ID where FACILITY_CODE = 'VNTR_AM'

E-BAM

sysLocCodeFileNameRegexGroups

^(.*)_(?'a'.*)_(.*)_(.*)$

DT_LOGGER.SYS_LOC_CODE

E-BAM

loggerCodeFileNameRegexGroups

DT_LOGGER.LOGGER_CODE

E-BAM

locationParamCodeFileNameRegex
Groups

^(.*)_(.*)_(?'a'.*)_(.*)$

DT_LOGGER_SERIES.
LOCATION_PARAM_CODE

E-BAM

dateTimeColumnHeaderRegex

^Time$

DT_LOGGER_DATUM.DATUM_UTC_DT; column header text is Time.

E-BAM

seriesNameColumnHeaderRegex
Groups

((?'a'.*)\(|(?'b'.*))

DT_LOGGER_SERIES.SERIES_NAME

E-BAM

seriesUnitColumnHeaderRegex
Groups

\((?'a'[^)]*)\)[^(]*$

DT_LOGGER_SERIES.SERIES_UNIT

 

2.Start the agent and copy the following file, named VNTR_AM_MV-1_PM10_06162019.csv, to folder C:\EarthSoft\Live\E-BAM.

 

 

Time,ConcRT (mg/m3),ConcHr (mg/m3),Flow (l/m),WS (m/s),WD (Deg),AT (C),RHx (%),RHi (%),BV (V),FT (C),Alarm,Type

10/23/2018 8:00,0.006,0.009,16.7,1.3,71,26.5,0,49,14,34.2,0,1

10/23/2018 8:30,0.006,0.009,16.5,2.1,79,29.2,0,50,14.1,34.6,0,1

10/23/2018 9:00,0.024,0.018,16.7,2.8,93,30.3,0,50,14,34.9,0,1

 

 

3.The file's name contains the DT_LOGGER.FACILITY_ID, SYS_LOC_CODE/LOGGER_CODE (and DT_LOGGER_SERIES.LOCATION_PARAM_CODE) fields. Find/add the logger row, for example:

 

LOGGER_CODE

LIVE_DATA_SOURCE

FACILITY_ID

SYS_LOC_CODE

MV-1

Folder://COMPUTER_NAME/
C:/EarthSoft/Live/E-BAM

DT_FACILITY.FACILITY_ID, where FACILITY_CODE = 'VNTR_AM'

MV-1

 

4.The file's first non-empty line is the header line (because headerLinesRegex is not defined); it contains the DT_LOGGER_DATUM.DATUM_UTC_DT column Time. The other columns are the series; they contain DT_LOGGER_SERIES.SERIES_NAMEs and SERIES_UNITs.

 

5.For each subsequent line:

a.For each series (each column, excluding Time column):

i.Find/add the series row for the SERIES_NAME.

ii.Add the datum row (e.g., for the first three series in the first data line):

 

SERIES_NAME

DATUM_UTC_DT

DATUM_VALUE

SERIES_UNIT

ConcRT

2018-10-23 08:00:00

0.006

mg/m3

ConcHr

2018-10-23 08:00:00

0.009

mg/m3

Flow

2018-10-23 08:00:00

16.7

l/m

 

The DT_LOGGER_SERIES.LOCATION_PARAM_CODE = 'PM10', for each of these series.