Windows data

This page is designed to help customers leverage the Forward Events integration within their Anvilogic account for FluentBit.

Pre-Reqs

• Anvilogic account

• Snowflake data repository connected to your Anvilogic account

• AWS CLI Installed

• FluentBit installed

Setting up FluentBit Config

1. Anvilogic will provide a S3 bucket and the corresponding access keys/ids (note these change for each integration) when you create a forward events integration in your Anvilogic deployment.

   1. 
2. Following the steps of the AWS CLI install, once you have done the installation correctly - Please run aws configure and paste in the access key and id provided. Once this is completed, validate that the credentials have been created - usually C:\Users\YourUsername.aws\credentials.

3. Once that has been validated, we need to create a system variable in order for fluentBit to read/use these credentials. To do so;

   1. Open the Start Menu and search for “Environment Variables.”

   2. Select Edit the system environment variables.

   3. In the System Properties window, click the Environment Variables button.

   4. Under System variables, click New.

   5. Enter the following:

      1. Variable name: AWS_SHARED_CREDENTIALS_FILE

      2. Variable value: C:\Users\YourUsername\.aws\credentials

   6. Next we need to configure fluentbit to read our logs and send them to S3. In this example, we will be ingesting the windows event logs. You can change what channels by simply adding or removing them.

      1. Please note, the bucket will be the bucket name/path.

         1. This could mean that it is sdi_customer_data-1 or -2 or -3.

[INPUT]
    Name         winlog
    Channels     Security, Application, System
    Interval_Sec 1

[OUTPUT]
    Name              s3
    Match             *
    bucket            avl-raw-prod-s3-221-24243202/sdi_custom_data-1
    region            us-east-1
    use_put_object    On
    Store_dir         C:\Windows\Temp\fluent-bit\s3
    s3_key_format     /$TAG/%Y/%m/%d/%H-%M-%S

Once you have pasted the above config into your fluentBit.conf file (typically located at C:\Program Files\fluent-bit\conf )

• NOTE: You can also edit or add any of your own customer parsers for logs by editing the parser.conf file at /etc/fluent-bit/

• Once you have edited your fluent-bit.conf, please restart the fluentBit application

1. You can now confirm that data has landed in your snowflake account.

   1. 

      

Please update the input section of this example config to fit your exact needs.