Using LightIngest on Ubuntu

It took me a while to figure out on how to install and use LightIngest on Ubuntu. So that’s why I created a small blog on how to do so.


Just execute the script below on Ubuntu to install LightIngest. Make sure to replace <version number> to the latest version, which can be found here:

sudo apt-get install unzip

sudo curl --location<version number>/ -O


chmod +x LightIngest

Now LightIngest is installed, it is possible to use it. For more information on how to use it, I refer to the Microsoft website.

Using LightIngest with managed identity

In my situation, I run an Ubuntu Azure Virtual Machine with a system assigned managed identity. After giving the system assigned managed identity permission to Azure Data Explorer and set the managed identity policy in Azure Data Explorer, I used the script below to start the ingestion:

sudo apt install python3

response=$(curl '<ADX cluster>.<location>' -H Metadata:true -s)

access_token=$(echo $response | python3 -c 'import sys, json; print (json.load(sys.stdin)["access_token"])')

./LightIngest "https://ingest-<ADX cluster>.<location>;Fed=True;UserToken=$access_token" -database:"<Database>" -table:"<Table>" -source:"https://<Storage Account><Container>" -format:"<File format>" -pattern:"*" -ingestionMappingRef:"<Table Mapping>"

Leave a comment

Your email address will not be published. Required fields are marked *

Exit mobile version