Making the Most of Present Information with Atlan
The Lively Metadata Pioneers sequence options Atlan prospects who’ve lately accomplished an intensive analysis of the Lively Metadata Administration market. Paying ahead what you’ve discovered to the subsequent knowledge chief is the true spirit of the Atlan neighborhood! So that they’re right here to share their hard-earned perspective on an evolving market, what makes up their trendy knowledge stack, revolutionary use circumstances for metadata, and extra.
On this installment of the sequence, we meet Robert Bayliss, Technical Supervisor, Information Platform from Racing and Wagering Western Australia, who shares how a contemporary knowledge catalog will probably be key to making sure their knowledge is seen, full, correct, and well timed for the enterprise companions they serve.
This interview has been edited for brevity and readability.
Would you thoughts describing RWWA and the way your knowledge crew helps the group?
Working for Racing and Wagering Western Australia (RWWA) is extremely distinctive, given we’re the controlling authority for thoroughbred, harness, and greyhound racing throughout our State, whereas we even have the accountability for off-course TAB wagering. We’re right here to supply a sustainable future to the Western Australia racing trade – an trade which is price $1.3 billion for the Western Australia financial system yearly. Nevertheless, racing is price rather more than its financial injection, its social advantages are immeasurable, particularly in regional cities, fostering neighborhood connection from Albany to Kununurra and in every single place in between.
Given how huge RWWA is, know-how performs a serious position in our day-to-day operations. Our knowledge platform crew is part of a bigger know-how crew, with our core duties and obligations targeted on making certain the information platform gives correct, well timed, full, and reliable knowledge to your complete RWWA crew.
May you inform us a bit about your self, your background, and what drew you to Information & Analytics?
I started my profession constructing data-driven functions, and actually loved making apps that enabled folks to do issues they couldn’t do earlier than. I moved into database growth, answer design, ICT challenge administration, after which, main an functions crew.
Lastly, I’ve moved to RWWA the place I lead the information platform crew and implement our knowledge technique. Finally, all my roles have been targeted on enabling folks and groups to realize issues that they couldn’t do earlier than. I consider knowledge to be an important enabler that anybody can use to enhance the best way they do issues.
What does your knowledge stack appear like?
With our supply knowledge unfold throughout a various know-how panorama together with Oracle, SQL Server and bespoke APIs, we needed to harness numerous applied sciences to realize our targets.This consists of AWS DMS for knowledge extraction, S3 for intermediate storage and Snowflake for Enterprise Intelligence and reporting. Each Python-based Lambas and DBT scripts orchestrated by means of Airflow full the transformation stage in our ELT course of.
Why seek for an Lively Metadata Administration answer?
Now we have plenty of knowledge that may assist our groups make well timed and significant selections, so the necessity to supply an Lively Metadata Administration Answer was to assist enhance the visibility and accessibility of our knowledge to everybody within the group, from people who work carefully with our knowledge engineers to those who help the animal welfare applications.
These stakeholders want entry to correct knowledge, and the information catalog was the device to try this. The elemental query was “How will we floor our knowledge, and the way will we make it correct and well timed so that folks can depend on it?”
Why was Atlan a superb match? Did something stand out throughout your analysis course of?
To be trustworthy, Atlan was the one device that made our knowledge engineering crew excited and additionally resonated extraordinarily nicely with the top customers. It made sense from a usability perspective. It was straightforward to visualise, it flowed and the simplicity of the expertise meant not too many clicks to get to the knowledge.
We have been actually targeted on the truth that our engineering crew was excited by it, and preferred the truth that we are able to see lineage and expose our knowledge units, present the way it’s all related, and likewise floor all of our knowledge high quality metrics, as nicely.
What do you propose on creating with Atlan? Do you could have an thought of what use circumstances you’ll construct, and the worth you’ll drive?
Information accessibility, in addition to surfacing knowledge observability metrics, are actually vital to us. It’s no use gaining access to knowledge if you’re uncertain of the completeness, accuracy, or timeliness. Atlan helps us to realize this by surfacing our knowledge high quality guidelines and observability metrics alongside our knowledge property.
Moreover, having a technique to floor knowledge classifications and key glossary phrases is a big profit to all of our groups, to assist convey a typical understanding and settlement on terminology.
It’s actually about ensuring that we’re leveraging our property. We’ve received massive volumes of information nevertheless it must be within the palms of those that can add worth with it.
The rest to share?
For me, it’s all the time in regards to the why. It’s, “Why are we really doing something?” The explanation for Atlan was all the time about how we get probably the most out of our knowledge. That was the problem, then we labored out what we may use to make that occur.
Picture by Philippe Oursel on Unsplash
