Jan 2Get started with Spark for Data Engineering in Azure Synapse AnalyticsThe Apache Spark engine has become more and more prevalent in the data engineering space. And for good reason — it enables data engineering at scale in the cloud, is welcoming to data professionals of various backgrounds (Python, Scala, SQL, Java, R) and works great in unison with Delta Lake. …Data Engineering3 min readData Engineering3 min read
Nov 7, 2022Common KQL features to know when working with Azure Data ExplorerGetting started with Azure Data Explorer (ADX) and Kusto (KQL) is fun but as with any language there is a learning curve. With this article I aim to showcase operators and functions that you’ll come across frequently! P.S. the example queries used will reference demo tables included with every ADX…Azure Data Explorer7 min readAzure Data Explorer7 min read
Oct 3, 2022How did I get started with Azure Data Explorer?Azure Data Explorer (ADX) has been gaining traction recently and for good reason. ADX makes it easy to analyse high volumes of data with the capability to handle near real-time data. As Microsoft puts it, “Azure Data Explorer is scalable, secure, robust, and enterprise-ready, and is useful for log analytics…Azure3 min readAzure3 min read
Sep 5, 2022Using the Spotify API for data-driven analysis of my playlists (Part 2/2)Hey there and welcome to the second part of using the Spotify API for a data-driven analysis of playlists. Last time we went through connecting to the API programmatically, and how to process the necessary data. …Power Bi10 min readPower Bi10 min read
Aug 15, 2022Using the Spotify API for data-driven analysis of my playlists (Part 1/2)Have you ever wanted to analyse the music in your Spotify playlists? Well, I know I did for a few months and I finally bit the bullet. These APIs provided by media services are not just an excellent exercise to practice Data Science skills, but also to get something valuable…Spotify9 min readSpotify9 min read
Jul 8, 2022Use your database to cancel Power BI refreshesSay you accidentally clicked the refresh button and now Power BI is hitting your database in the middle of the day. There is no option to stop it in the Service at the time of writing, and the option provided by Microsoft is to use its API to do the…Sql4 min readSql4 min read
Published inMLearning.ai·May 30, 2022Forecasting SP500 stocks with XGBoost and Python Part 2: Building the modelWelcome to part 2 of the “Forecasting SP500 stocks with XGBoost and Python” series, a two-part series where I take you through creating a basic XGBoost model for time series forecasting. The goal with this series is to take you through how I created a basic forecasting model, based on…Python9 min readPython9 min read
May 16, 2022Forecasting SP500 stocks with XGBoost and Python Part 1: Sourcing the dataA while back I dabbled with the question “how little effort can I put into building an XGBoost model for a financial dataset?”. This question didn’t pop out of thin air. I was looking to get some experience with a Machine Learning model for timeseries forecasting and I was discussing…Python3 min readPython3 min read
Published inLevel Up Coding·May 10, 2022Run unpublished Spark notebooks in Azure SynapseYou can now reference unpublished Spark notebooks in Azure Synapse Analytics! Previously, you could only reference a notebook if you had already published it into the Synapse workspace. In other words, if the notebook only existed in the Git repository then you had no chance to call that notebook from…Azure3 min readAzure3 min read
Published inPython in Plain English·Apr 26, 2022Getting Started with Pygame: Animate Falling GemsWhat is Pygame? It is a fun library in the Python ecosystem which you can use to build 2D games but also simple 2D animations. — Today I want to show you how to code an endless animation of falling gems — totally not inspired by the currency from The Legend of Zelda :)Python4 min readPython4 min read
