Data Concept: Idempotence

Idempotentity is not only useful in the design of data pipelines, but also as a system-wide concern, since if one step is not idempotent, the entire quality of the results will be affected. Let's look at a practical example.

Diagrams as Code in your blog with Mermaid

So the people that know me might remember that I have a sort of obsession with diagrams. It helps me think and evolve my thoughts. I also love how git works and the importance of having code that can see its changes across time. So it's no surprise

SQL + DataOps = SQLMesh?

DBT is valuable but lacking. SQLMesh promises enhancements like unit tests, environments, airflow support, change plans, and CI/CD automation, providing a better development experience. Let's see its features.

Dangers of recursion (python vs rust): Fibonacci case

Fibonacci sequence is commonly used to show recursion, but may not be the best algorithm to compare languages and performance, since the recursive implementation is not the optimal one and languages like python will outperform rust or C.

Spark and HugginFace: Named Entity Recognition for aircraft ownership

How to use one of the best AI/ML libraries (hugginface) with Spark? Let's try it

Scotland structure

Some pictures taken in Scotland. Use them as you please under CC BY-NC-SA and if used in public (blog, presentation , news article...), cite me or this web/post. Can't be used to train AI/ML algorithms.

Troubleshoot pyspark Py4JNetworkError: Answer from Java side is empty

The pyspark library communicates to a spark driver. If the driver dies, we'll obtain some odd error messages that will not directly indicate the root cause.

reMarkable: a paper tablet still in beta

After 4 years owning two reMarkable, it's time I write the good, bad an ugly things of it.

Scottish landscapes

Some pictures taken in Scotland. Use them as you please under CC BY-NC-SA and if used in public (blog, presentation , news article...), cite me or this web/post. Can't be used to train AI/ML algorithms.

Troubleshoot: NoClassDefFoundError when reading with Spark Avro

Just a quick post on an error you  might find when using apache spark and Avro due to a version mismatch. Software Version Spark 3.3.2, 3.3 spark-avro 2.12:3.4.0 I've you have an older version of spark (latest as of now is

Asciidoctor: the best way of writing technical presentations

If you write code and you are bored when someone mentions PowerPoint, this asciidoctor slide template is for you. Write code and get animations for free!

Build beautiful presentations with asciidoctor and reveal.js

I've been a long time looking for an excuse to really use asciidoctor to build presentations. Now that I have the opportunities, I've been enjoying it like a child. For my memory, and for yoy, if you want to use it, here are my tips. This