skip to Main Content

The DataOps Blog

Where Change Is Welcome

You Graduated! What’s Next? Career Advice for Data Engineers

By Posted in StreamSets News May 25, 2021

Welcome, new grad, and congratulations – you did it! Getting your degree is the first of many achievements you’re sure to accomplish throughout your professional career. As you contemplate joining “the real world,” you’re probably curious about what’s in store for you. 

career advice for data engineersYou’re getting advice left and right, and separating the wheat from the chaff isn’t always easy. Don’t worry; we’ve checked out the advice online and asked people at StreamSets what they wish they’d known at the start of their careers. Which brings us to our first advice as you embark on your IT career:

Career Advice Tip #1: Write Your Own Story

“I wish someone had told me: find a way to listen to yourself more than you listen to your family and friends,” said Judy Ko, chief product officer, StreamSets. Yes, our first advice is: Don’t worry so much about the advice you get! 

Raji Narayanan, head of product management at StreamSets, whose daughter graduates from high school next year, advises, “Do not second guess yourself. Dive in and try different things. If you think, can I really do this? Ask yourself, why not?” Believe in yourself, follow your gut, and you probably won’t end up too far off the right path. Because there is no one right path. Most everyone we know followed a long, winding, and unexpected career path to where they stand today. 

Vu Trang started her career as a software engineer almost two decades ago before moving into data science and data analytics roles. She joined StreamSets as head of technical support before becoming StreamSets’ first official data person. Today, she is head of business analytics, where she does her own data engineering. 

“Most data engineers I know just kind of fell into it,” said Vu. “They started in data science or analytics and had to do their own data engineering work.”

This leads us to our next piece of advice…

Career Advice Tip #2: Learn All You Can – and Keep Learning!

“When you get out in the world of business, what people think is important is very different from what you experienced at a university,” said Susannah Medley, Software Engineer IV, StreamSets. “You may think if you are really, really good at one thing, you’re set. But that’s less important than diversifying your knowledge. Talk to people outside of your department and people doing similar things outside of your company. ” 

Data engineering requires a broad set of skills. Just check out the data engineers interviewed for our Sources and Destinations Podcast. Here’s a summary of what we found in our research:

Data engineering calls for strategic thinking:Beyond the really interesting technical challenges that come with working with big data, there are lots of opportunities to think about higher-level domain challenges as a data engineer,” says Dhevi Rajendran, Data Engineer, Netflix.

Serious communication skills: Grace Ge, Associate Director of Data Engineering at MobilityWare says, “I work very closely with our CTO and CRO, along with the product, marketing, monetization, and engineering teams. I do my best to truly understand each departments’ requirements and business goals so that I can design the most efficient systems possible.” 

And, of course, lots of tech chops: “In order to be a strong data engineer, you need a diverse skill set. This includes programming, SQL, cloud, data and software engineering, design, and data science…It is essential to be curious and to learn about state-of-the-art technologies,” shared Amogh Antarkar, Data Engineer At HEB.

StreamSets Director of Evangelism Dash Desai advises new graduates to start using popular tools and technologies since most have free trial versions. “Consider getting certified from the leading cloud vendors/providers AWS, Google, Microsoft and also in technologies like Spark,” he said. “And familiarize yourself with machine learning and data science because most tech companies expect data engineers to be well versed in these disciplines today.”

Vu puts another twist on her advice to learn as many technologies as you can. “I can 100% guarantee you that what you learn in school will not prepare you 100% for any job you get.”

Career Advice Tip #3: Make Yourself Useful to the People Around You

Finally, the advice to be of service to others is one that has never failed us yet. And it has a way of coming back around over the years. This blog by Dash about analyzing COVID-19 Open Research Datasets is an example of using data engineering for public benefit. And we wrapped up several ideas in this blog on Data Engineering for a Purpose

Vu suggests, “Contribute to an open-source project. There are tons of them out there. You can donate time to a favorite cause, help the world, meet people – and it looks great on your resume.” 

Gaining experience working collaboratively will serve you well in your data engineering career. Lena Bartell, Lead Data Scientist at Morse Corporation, shares that their “data engineers (or any particular discipline) are not separated off into their own group. Teams are built with engineers who have a mixture of skills, so you’re always working with people from both similar and different backgrounds. This is great for building a diverse and talented project team as well as allows all of us to gain knowledge on new topics.”

Lorna Mitchell, Developer Advocate at Aiven and software engineer, author, speaker, teacher, and open source enthusiast found her way to her passion by “building odds and ends for different student societies when I was at university.” Lorna says, for her, open source isn’t about a product being free. Instead, it’s about being able to go in and fix a product or add a new feature. Learn more about Lorna’s work with APIs, thoughts on under-appreciated coding paradigms, and more in the Sources and Destinations interview (transcript).

Career Advice Tip #4: Go Forth and Conquer

You’re young; people will want to help you,” suggests Mary Berg, director of SI and channels, StreamSets. “Your job is to let them.” 

 It’s not always easy to accept help, especially when you’re ready to take on the world. But one thing we’ve learned is no one gets through their career alone. 

Those with the strongest networks have people to call on in good times and bad. And weaving your way through your career path is easier the more you explore.

“The hard part of a job isn’t really solving the problem,” Vu shared. “The hard part is identifying the problem. Working with your cross-functional stakeholders, partners, and users can really increase the velocity of finding the problem and then solving the problem. It can also open doors and open your eyes to opportunities you probably never learned about in school, where you’re solely focused on engineering. For example, meeting with folks in sales and marketing could open doors for other career paths in consulting or sales operations that also require that analytical mindset.” 

As a data engineer to be, the professional world is your oyster. Go get ‘em! 

Year of the Data Engineer

why data engineering for climate change earth picture

Back To Top

We use cookies to improve your experience with our website. Click Allow All to consent and continue to our site. Privacy Policy