title: Elasticsearch event ingestion pipelines
body: <p>New to Security Onion, so I am trying to understand how Elasticsearch ingestion works.</p>
  <p>I currently have an agent on a server sending some json (or CSV) formatted events to our Security Onion, and I want to ingest them in Elasticsearch. I can either send them directly to Elastic, or send to Logstash and from there to Elastic.</p>
  <p>First of all, it is not clear to me how Elastic decides which pipeline to use?</p>
  <p>Second, I want to make sure that the fields that the target fields are already existing in the database? For example, if in the json there is a field <code>event_destinationProcess</code> - how I find out what is the corresponding name of the field that Elastic is already using (e.g. <code>destination.process</code>)?</p>
  <p>This is an example of event I want to parse:</p>
  <pre><code>{&quot;EventTime&quot;: &quot;2021-11-30T00:12:12.899263-05:00&quot;,&quot;Hostname&quot;:&quot;MYSERVER&quot;,&quot;Keywords&quot;:&quot;9223372036854775808&quot;,&quot;EventType&quot;:&quot;INFO&quot;,&quot;SeverityValue&quot;:2,&quot;Severity&quot;:&quot;INFO&quot;,&quot;EventID&quot;:11,&quot;SourceName&quot;:&quot;Microsoft-Windows-Sysmon&quot;,&quot;ProviderGuid&quot;:&quot;{5770385F-C22A-43E0-BF4C-06F5698FFBD9}&quot;,&quot;Version&quot;:2,&quot;TaskValue&quot;:11,&quot;OpcodeValue&quot;:0,&quot;RecordNumber&quot;:6682123,&quot;ExecutionProcessID&quot;:9080,&quot;ExecutionThreadID&quot;:13784,&quot;Channel&quot;:&quot;Microsoft-Windows-Sysmon/Operational&quot;,&quot;Domain&quot;:&quot;NT AUTHORITY&quot;,&quot;AccountName&quot;:&quot;SYSTEM&quot;,&quot;UserID&quot;:&quot;S-1-5-18&quot;,&quot;AccountType&quot;:&quot;User&quot;,&quot;Message&quot;:&quot;File created:\r\nRuleName: -\r\nUtcTime: 2021-11-30 05:12:12.899\r\nProcessGuid: {CA401AF7-1F55-6184-4700-000000009600}\r\nProcessId: 4148\r\nImage: C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe\r\nTargetFilename: C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\Health Service State\\Monitoring Host Temporary Files 8\\1470202\\Microsoft.Windows.Server.DNS.WINS.Discovery.vbs\r\nCreationUtcTime: 2021-11-30 05:12:12.899\r\nUser: NT AUTHORITY\\SYSTEM&quot;,&quot;Category&quot;:&quot;File created (rule: FileCreate)&quot;,&quot;RuleName&quot;:&quot;-&quot;,&quot;UtcTime&quot;:&quot;2021-11-30 05:12:12.899&quot;,&quot;ProcessGuid&quot;:&quot;{CA401AF7-1F55-6184-4700-000000009600}&quot;,&quot;ProcessId&quot;:&quot;4148&quot;,&quot;Image&quot;:&quot;C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe&quot;,&quot;TargetFilename&quot;:&quot;C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\Health Service State\\Monitoring Host Temporary Files 8\\1470202\\Microsoft.Windows.Server.DNS.WINS.Discovery.vbs&quot;,&quot;CreationUtcTime&quot;:&quot;2021-11-30 05:12:12.899&quot;,&quot;User&quot;:&quot;NT AUTHORITY\\SYSTEM&quot;,&quot;EventReceivedTime&quot;:&quot;2021-11-30T00:12:14.821147-05:00&quot;,&quot;SourceModuleName&quot;:&quot;sysmon_event&quot;,&quot;SourceModuleType&quot;:&quot;im_msvistalog&quot;}
  </code></pre>