POS master data updates over signal watchers, to execute a list of tasks (for now only import and/or export via datainterchange) depending on newly created files. This so called trigger-file will be deleted after the execution of all the tasks (or actions) weither they are be executed sequentially or parallel.
A client wishes to export a certain amount of data (sales, reports, master data...) on a specific working day (or not) at a certain time, either once or several time on this date. Therefore, it should now be possible to define such characteristic inside a model.
import declarations - Used to import external models or other Java classes.
watcher - define a watcher which monitors a directory.
filemask - define a group of target files to watch and all detail about the watcher.
filename - define a target file to watch and detail about the watcher.
scheduler - define the simple or complex schedulers for executing tens, hundreds, or even tens-of-thousands of jobs.
With the keyword package followed by a qualified name, you define the root element that contains all the other elements. A model can contain multiple packages.
filemask <file mask> . . .
| filename <file name> . . .
| scheduler <scheduler name> . . .
one or more watcher/scheduler can be defined in the same package.
2 types of watcher can be defined, they are file mask watcher and file name watcher.
The trigger policy defines how the execution of its list of tasks be triggered: either file based (the creation of a file) or scheduled plan based.
Currently we only support the export and import functions of existing Datainterchange Units from the DatainterchangeDSL as the possible watcher's/scheduler's tasks to be executed.
The properties of DatainterchangeDSL are saved in the Datainterchange Configuration Files which are generated with the DatainterchangeDSL.
The Datainterchange Configuration Files are default saved in C:\Users\<user name>\.osbee. Per Datainterchange Group, a <Datainterchange Group name>Config.xml file will be generated automatically. This file path can be changed in Eclipse--> Preferences--> OSBP Application Configuration -->External Data Source --> Datainterchange Setting.
The corresponding watcher's and scheduler's properties defined in SingalDSL will be added as new entries into the corresponding Datainterchange Configuration File (<Datainterchange Group name>Config.xml) automatically after saving the SignalDSL file, like <entry key="<package name>.WithMaskWatcher.filemask"><file mask></entry> , or <entry key="<package name>.WithMaskWatcher.filename"><file name></entry> , or <entry key="<package name>.<scheduler name>Scheduler.<scheduler type>.<scheduler property>"><property value></entry> . The <scheduler type> can be hourlyscheduler,dailyscheduler,monthlyscheduler,weeklyscheduler and cronscheduler; the <scheduler property> can be minute, hour, dayofweek,dayofmonth, and expression.
In cases the needed datainterchange properties are not available in the Datainterchange Configuration File, or the Datainterchange Configuration File doesn't exist, the default definition of the properties from the DatainterchangeDSL file will be used.
Currently it is not possible to monitor changes on a (web)url-based directory (destination). You may however define a watcher with a url-based directory and set schedulers to import data from it.
Watcher/scheduler generates watcher/scheduler jobs, they will be stored into a queue and processed by the watcher/scheduler job handler. Keyword sequential and parallel determinate how the jobs will be executed. As same as the literally meaning, Keyword sequential means all tasks/jobs will be sequentially executed; Keyword parallel means all tasks/jobs will be parallel executed. If it is not defined, sequential will be take as the default value.
► Following are the example predefined datainterchange definition and the corresponding configuration files which will be used in the SingalDSL definition example in this documentation.
map brandtypecd to "brandtypecd" map bsin to "bsin" mapBlob bsinmedia to "bsin" extension "jpg" path "C:/POD/brands/brand" mimeType jpg map brandnm to "brandnm" map brandlink to "brandlink" map id to "id"
the Datainterchange Group name is Brands, and the Datainterchange Units are Brandowner,Brandtype,Brand,Gtin and Pkgtype.
2 entries of import/export directories are generated for each Datainterchange Unit and an entry of Max-Parallel-Threads-Count value is generated for this Datainterchange Group. All values in this file can be modified manual.
Watcher jobs are generated by a watcher, they will be stored into a queue and processed by the watcher job handler.
A target file is the file which is generated inside the monitored directory, whose filename (+extension) matches the file mask or the file name.
The datainterchange file used for import processes will be deleted at the end of a successful import, if it is explicitly set in its datainterchange definition. See Datainterchange DSL for more information.
In case the target file is at the same time used in datainterchange as the import file, it will be deleted at the end of the execution of the watcher job (parallel or sequential), even if it's not set in the datainterchange definition.
If the import process is failed, at the end of the import process, the corresponding import file will be renamed with form FAILEDIMPORT+< actual date formated >+Original file name.LOCKED (as the definition in the following blog post).
The keyword applyon has been introduced in order to define per watcher, which directory has to be monitored for the triggering events: creations of target files in this directory, which match the given file mask or file name.
The watcher-monitored directory corresponds to the import-path ( in configuration file after keyword entry key ="<datainterchange uinit >-import" ) of the Datainterchange Unit marked with the keyword applyon as the watcher tasks in the SignalDSL file.
If the DatainterchangeDSL Configuration File (per Datainterchange group a file, which contains also the SignalDSL properties) exists, the needed import-path property of the Datainterchange Unit exists and its value is valid, then this file path will be used to monitor. Otherwise, the default path of the Datainterchange Unit defined in the corresponding DatainterchangeDSL definition will be used as the monitored directory.
The watcher job queue is used to secure the sequential execution of each watcher jobs (task/list of tasks).
The jobs generated at the same time from the same file mask watcher can be called as a job group. A grouping of jobs from the same watcher is actually taking place in order to better make use of multi-threading functionalities for the watchers which are allowed a parallel execution of their jobs (only one for a file mask watcher).
Grouping the execution of jobs from the same watcher is mostly useful in the case of parallel execution of the watcher jobs. Currently this concerns watcher with file mask definition, whose jobs execute only one task. The file mask watcher can be used for importing a huge amount of data (independently / orthogonally) split over numerous file into os.bee software. Those file could be processed parallel with a same file mask watcher. The amount of parallel executed jobs is limited and determined by the property Max-Parallel-Threads-Count, which can be found and set in the corresponding DataInterchange Configuration File.
Please note that the watcher job from a file name watcher defined with sequential execution type will process its list of tasks in only one dedicated Thread instance; the watcher job from a file name watcher with parallel execution type can execute its tasks in several available threads (not exceeding the maximum thread count).
The maximum thread allowed to be parallel processed (max-parallel-thread-count) is defined in the Datainterchange Configuration File, after keyword entry key="Max-Parallel-Threads-Count" for each Datainterchange Group. In the run time, the maximum value of all max-parallel-thread-count from all Datainterchange Groups will be used to set the runtime max-parallel-thread-count.
The current allowed maximum value is 15, and default value is 5. If this value could not be found from any Datainterchange Configuration File for any Datainterchange Groups, the default count will be used to set the runtime max-parallel-thread-count.
We provide the possibility to set a file mask to extend the further specify (and identify) on a file name basis, the defined task will be executed when the matched files creations occur in the monitored directory.
One or more matched target-files can be found in the same monitored directory each time.
The directory to monitor is determined by the only one Datainterchange Unit in file mask watcher.
One or more watcher jobs can be found in the same job group of watcher job queue.
For each watcher job, only one watcher task can be defined.
for parallel execution : at most max-parallel-thread-count threads of jobs in the same job group can be executing at the same time.
for sequential execution : the jobs in the same job group will be executing one by one.
the file mask watcher WithMask monitors the import directory of Datainterchange Unit Gtin from Datainterchange Configuration File, when the file names match the mask "Gtin*.csv" (Gtin123.csv, Gtinabcd.csv...) occur, the watcher jobs with the same task import Gtin will be added to the same watcher job group into watcher job queue and the jobs in the same job group will be executed parallel with at most max-parallel-thread-count threads.
A specified file name can be defined as the target file, and a list of tasks will be executed when the matched file creation occurs in the monitored directory.
Keyword file name is used to define the identification criteria (<filename>.<extension>), you define the name of file, which have to be created and match the given file name, as trigger event.
Only one matched target-file will be found in the monitored directory each time, so only one watcher job can be found in the same group of watcher job queue.
The directory to monitor is determined by the Datainterchange Unit which marked with the keyword applyon in filename watcher.
For each watcher job, one or more watcher tasks can be defined.
for parallel execution : at most max-parallel-thread-count threads of tasks in the same job can be executing at the same time.
for sequential execution : the tasks will be executing one by one.
the file name watcher WithName monitors the import directory of Datainterchange Unit Gtin from Datainterchange Configuration File, when the file named "brands.csv" occurs, the watcher job with task list import Brandowner ... export Brand will be added to the watcher job queue, the tasks of the job from the same task list will be executed parallel with at most max-parallel-thread-count threads.
The scheduler functionalities are built upon the Quartz Job Scheduler Framework.
Scheduler jobs are timely executed at run-time based upon their scheduler definition.
You are able to define an hourly, a daily, a weekly or a monthly schedule plan, to set when to execute the list of tasks of a Scheduler. Additionally, you can also be more straightforward and define a more complex schedule plan with cron-expressions.
More information about how to build cron-expressions can be find here.
The minute entry has to be between 0 and 59.
the scheduler HourlyJob will be executed every hour at the 55th minute.
The hour entry has to be between 0 and 23.
the scheduler DailyJob will be executed every day at 18:10.
Day Of Week Enum is including sunday, monday, tuesday, wednesday, thursday, friday and saturday.
the scheduler WeeklyJob will be executed on every saturday at 6:15.
The day of month entry has to be between 1 and 31.
the scheduler MonthlyJob will be executed on every month the 28th day at 20:45.
the format of cron expression is defined as a string like "<second> <minute> <hour> <day of month> <month> <day of week>".
The value of second entry has to be between 0 and 59.
The value of minute entry has to be between 0 and 59.
The value of hour entry has to be between 0 and 23, or * for every hour.
The value of day of month entry has to be between 1 and 31, or * for every day.
The value of month entry has to be between 1 and 12, or * for every month.
The value of day of week entry has to be a value of SUN, MON, TUE, WED, THU, FRI, and SAT, or ? for every week day, MON-SUN for every workday.
Do the job every day (count from every month the 29th till the end of the month), every 5 hours (count from every day 10 o'clock till the end of the day), every 5 minutes (count from the 53th minutes of every hour till the end of the hour).
this scheduler CronJob will be executed every 5 minutes.
This page was last modified on 12 November 2018, at 12:34.