Driving Self-service and Enhancing DataOps with Atlan
The Lively Metadata Pioneers collection options Atlan clients who’ve not too long ago accomplished an intensive analysis of the Lively Metadata Administration market. Paying ahead what you’ve discovered to the subsequent information chief is the true spirit of the Atlan group! So that they’re right here to share their hard-earned perspective on an evolving market, what makes up their fashionable information stack, revolutionary use instances for metadata, and extra!
Within the first interview of this collection, we meet Heidi Jones, information evaluation and program administration extraordinaire, who explains the historical past of Docker’s information workforce, how they evaluated the market, and the way they’ll use Atlan to assist their colleagues drive one of many world’s finest developer experiences.
This interview has been edited for brevity and readability.
Would you thoughts describing Docker and your information workforce?
Docker is a platform designed to assist builders construct, share, and run fashionable functions. We deal with the tedious setup, so builders can concentrate on the code.
Information professionals at Docker help quite a lot of completely different departments. So we have now a core information workforce with engineers and analysts, after which we even have information engineers and analysts that help the main features of Docker, comparable to Advertising and marketing, Gross sales, the completely different merchandise at Docker, Finance, et cetera.
A number of skilled information engineers and analysts who’ve joined Docker, have solely began within the final 9 months or so. So we’ve had fairly a little bit of progress on the information workforce, and are actually at that stage the place we’re making an attempt to put money into good processes. That method, our information workforce can be sure that everybody at Docker has the information that they should do their jobs, and might finally assist builders do theirs.
And the way about you? May you inform us a bit about your self, your background, and what drew you to Information & Analytics?
I believe the primary motive I’ve been drawn to information and analytics is as a result of I similar to having the ability to reply individuals’s questions.
I got here into information evaluation by means of a non-traditional route. I’ve been at Docker for about six months now, however I’ve been within the information area for a couple of decade. It began with Excel and offering insights through spreadsheets, as much as PowerBI utilizing Snowflake, that sort of factor.
So I used to be at all times a knowledge analyst, however then additionally a mission supervisor. And so what I do at Docker combines each of these. The information of knowledge and the workflows required to get good information and supply good insights, and likewise the mission administration and operations aspect of it. All of it permits information professionals to concentrate on what they do finest, which is modeling information and offering insights with out being blocked by something that has to do with workflow.
What does your stack appear like? Why did you want an energetic metadata resolution?
We ingest information from quite a lot of sources in a number of other ways, relying on the supply. After which our information warehouse stage is Snowflake. Our modeling layer is dbt, that’s the place we do modeling and transformation. After which our most important BI device is Looker, that’s the place we do visualization and evaluation.
We have been only a one-person workforce not too way back. So all of that information work was on one particular person’s plate, together with documentation and understanding information sources. That’s fairly a bit for one particular person.
Numerous that burden has been unfold out throughout a number of individuals on the workforce by now. However we’re making an attempt to maneuver away from, “Oh, let me go ask my favourite information particular person,” towards, “I can go examine this device and I do know there’s an authorized information asset.”
And so, due to our stack, we have been drawn to Atlan due to issues just like the Looker Chrome extension plugin, the dbt integration, that sort of factor. As a result of proper off the bat we have been in a position to say, “Okay, any descriptions we put in our dbt layer will routinely be uncovered in Atlan.”
So non-engineering customers who need to know what the information means can go straight to Atlan and see what’s being executed within the modeling layer.
Did something stand out to you about Atlan throughout your analysis course of?
Atlan is a really cool device that has suite of options that we have been on the lookout for, however the differentiator actually got here right down to the individuals at Atlan.
You demonstrated very competent understanding of the issues within the information area and likewise very mature buyer help. We might inform that your help was not simply one thing you have been promising for us, however one thing that you just have been already actively doing with different clients.
We knew that it might be an actual partnership and that the shopper help org was ready to help the wants of a company like ours. And that maturity stood out to us once we made our choice.
However then once more, additionally the options like Playbooks, the integrations that I’ve already talked about with dbt, with Looker, and simply the fixed innovation as effectively that we have been in a position to observe even throughout the analysis processes, which I imagine took us about two months.
There have been a number of improvements and releases that occurred throughout that point interval and we might see the cadence the Atlan was on to repeatedly enhance. All of these have been promoting factors to us.
What do you plan on creating with Atlan? Do you could have an thought of what use instances you’ll construct, and the worth you’ll drive?
Our largest worth that we’re making an attempt to drive with Atlan is to make it possible for professionals at Docker can get the knowledge they want in regards to the information that they should do their jobs.
We need to transfer in direction of self-serve analytics and permit each information professionals, and people who simply need to have the ability to use information extra freely of their work, to have the ability to achieve this with out having to get into all the SQL and technical particulars of the information.
They know they’ll belief the information set, they know they’ll belief the information that they’re taking a look at, they usually can go forward and make their choices. Finally, it ought to assist us help our mission of delighting builders, and creating instruments that they take pleasure in utilizing.
We’ll be supporting that with Atlan, and likewise supporting our information engineering and analytics groups. They should have extra supported and standardized workflows, in order that they’ll concentrate on modeling, actually digging in and doing what they do finest with information.
Did we miss something?
That’s query. I believe how we found Atlan was attention-grabbing. I’ve been following Prukalpa, really, for a few years simply as a knowledge skilled, simply form of watching Atlan.
And so after I joined Docker, they have been already taking a look at information catalog instruments, however hadn’t been taking a look at Atlan but. And I mentioned, “Effectively, how about Atlan? Ought to we take a look at Atlan as effectively?”
So one of many first issues I did at Docker was to begin up that dialog, and the explanation why I did that’s as a result of I had appreciated studying what she mentioned in these areas. Concerning the causes we’d like information catalog instruments, and past only a catalog, the way it could possibly be a part of information operations. And that piece of it actually had spoken to me over time.
And we noticed some spectacular instruments. It’s a burgeoning area. There’s some nice instruments on the market. However I’m glad that we additionally checked out Atlan as a result of finally it had mixture of what we would have liked at Docker.
Picture by Annie Spratt on Unsplash
