Tom Lous
I'm a Data & ML Software Engineer
I develop Scala and Python software that
runs on a Spark cluster or dockerize functional scala microservices to run on a Kubernetes cluster.
I'm proficient with many tools concerning setting up a big data ingestion & processing pipeline in
the cloud and deploying the results via a scalable API.
I'm also skilled in cleaning & analyzing huge amounts of data, followed by training, validating &
testing machine learning models and deploying them in production.
Experience
Senior Data Engineer
Building streaming data products with on Azure
Python, Kafka, dbt, Snowflake, Rust.
Senior Software Engineer
Building streaming data products with ZIO 2, Scala 3, Kafka and Cassandra on Azure Kubernetes
Scala, ZIO, Kafka, Azure, Kubernetes.
Lead Data Engineer
Leading the data factory team and implementing scalable data ingestion solutions for a data mesh architecture using Spark, Scala, Databricks, Kafka and Kubernetes (OpenShift)
Scala, Spark, ZIO, Azure, Kubernetes, Databricks.
Principal Engineer
Part of the Architecture Chapter
Spark, Python, AWS, Airflow, Ansible, Snowflake & Hadoop development.
Lead Data Engineer
Part of the Agile Hub
Spark, Scala, Azure, Kubernetes, Airflow, Terraform & Hadoop development.
Machine Learning Engineer
Part of the Advanced Analytics Platform (AAP) and Technical Passport (TP)
(Py)Spark, Hive, Oozie & Hadoop development.
Big Data Engineer
Data ingestion as a service (Kafka, Hadoop, Kubernetes) @ eBay's PE (Platform Engineering) Team
Spark, Scala, Flink, Hadoop, Cassandra, Kafka & Machine Learning @ eBay's CDATA (Central Data) Team
Big Data & Machine Learning Software Engineer
Building fully automatic data ingestion & processing pipelines in the cloud with Scala, Spark and Airflow. Enrichment via machine learning and deployment via API on top of Elasticsearch in Kubernetes
« 15 years of diverse tech related jobs »
In the past I've been a PHP web developer, MySQL database admin, Linux system engineer, IT
Manager, Team lead, Frontend developer, iOS Software developer, etc etc.
Less relevant to the tools I'm using now, but it does make me an experienced well rounded
developer.
Check LinkedIn for the
details, or read my
story
Speaker Bio
Building & Deploying Spark Applications
Deploying Apache Spark Jobs on Kubernetes with Helm and Spark Operator
Apache Airflow & Apache Spark data pipelines in the cloud
Building a Data Ingestion & Processing Pipeline with Spark & Airflow
Published Articles
CI/CD for Data Engineers
Deploying Apache Spark Jobs on Kubernetes with Helm and Spark Operator
Record linking with Apache Spark’s MLlib & GraphX
Re-becoming a developer
Installing Octave on Mac OS X Mountain Lion
Skills
Open Source contributions
- GCP's Spark on Kubernetes Operator - Contributor - Spark on Kubernetes via Operator
- Spark on Kubernetes Operator's Helm Chart - Contributor - Helm Chart of Spark on Kubernetes via Operator
- Strimzi Kafka Operator - Contributor - Apache Kafka running on Kubernetes and OpenShift
- Scalafiniti - Maintainer - Scala SDK wrapper around Datafiniti API
- http4s-rho - Contributor - Self documenting (swagger) DSL for http4s web server
- elasticsearch-client - Contributor - Elasticsearch client for Scala
- solr-schema-dataimport-generator - Maintainer - Generate solr config based on cetralized template
- « Many more OSS repos » - Mostly coursera related
Interests
Apart from being a software engineer, I organize a regular meetup for data enthousiasts in Rotterdam and surroundings, called Data Driven Rijnmond
I also spend a lot of my free time at night learning new things using MOOC like Coursera, resulting in 34 certifications
In my free time I enjoy spending time with my wife, daughter and son. I run, swim, play tennis and golf and get in shape via a personal trainer. I also hold 1st dan (shodan) in Aikido.
I love puzzles and attempt to escape from about 4-5 escape rooms every year.
When forced in doors I like to read, and watch Youtube (My favorite channels: PBS Space Time, Mark Rober, Numberphile, SmarterEveryDay, 3Blue1Brown, standupmaths, minutephysics, CPG Grey, Kurzgesagt, Veritasium, Critical Role, Simone Giertz). I also play board games or catch up on some movies / series
Links
Stack Overflow
Github
Links
Availability
starting October 2024
GraphIQ
GraphIQ
Spinel 7
2651RV Berkel en Rodenrijs
info@graphiq.xyz
+31645528510
KvK (CoC): 57477574
BTW (VAT): NL001703498B23
Bank (IBAN): NL15INGB0007044957
Swift (BIC) : INGBNL2A