Senior Data Engineer

Medium


2 months ago

03/22/2019 16:56:17

Category: Software Dev


At Medium, words matter. We are building the best place for reading and writing on the internet—a place where today’s smartest writers, thinkers, experts, and storytellers can share big, interesting ideas; a place where ideas are judged on the value they provide to readers, not the fleeting attention they can attract for advertisers.
 
We are looking for a Senior Data Engineer that will design, build, ship and maintain our business critical Data Platform. In this role you will lead development of both transactional and data warehouse designs mentoring our team of cross functional engineers and Data Scientists. You'll also design, implement and tune tables, queries, stored procedures, and indexes.
 
At Medium, we are proud of our product, our team, and our culture. Medium’s website and mobile apps are accessed by millions of users each day. Our mission is to move thinking forward by providing a place where individuals, along with publishers, can share stories and their perspectives. Behind this beautifully-crafted platform is our engineering team who works seamlessly together. From frontend to API, from data collection to product science, Medium engineers work multi-functionally with open communication and feedback.

What Will You Do

  • You’ll work on high impact projects that improve data availability and quality, and provide reliable access to data for the rest of the business
  • Design, architect and support new and existing data and ETL pipelines and recommend improvements and modifications.
  • Create optimal data pipeline architecture and systems.
  • Assemble large, complex data sets that meet functional and non-functional business requirements.
  • Be responsible for ingesting data into our data warehouse and providing frameworks and services for operating on that data including the use of Spark.
  • Analyze, debug and correct issues with data pipelines
  • Communicate strategies and processes around data modeling and architecture to multi-functional groups and senior level management.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Spark and AWS technologies.
  • You will build widely used data pipelines and tools making critical business data available to other teams.
  •  

About You

  • You have at least 5 years of experience implementing complex ETL pipelines preferably in connection with Hadoop or Spark.
  • You have lots of experience writing complex SQL and ETL processes
  • You have exceptional coding and design skills, particularly in Java/Scala and Python.
  • You've worked with large data volumes, including processing, transforming and transporting large-scale data
  • You have hands-on experience with AWS and services like EC2, SQS, SNS, RDS, Cache etc.
  • You have a BS in Computer Science / Software Engineering or equivalent experience.
  • You have knowledge of Apache Hadoop, Apache Spark (including pyspark), Spark streaming, Kafka, Scala, Python, and similar technology stacks
  • You have a strong understanding & usage of algorithms and data structures. 
  •  

Nice To Have

  • Spark data pipeline and or streaming experience
  • Redshift knowledge and operational experience
  • Machine Learning expertise
  •  
At Medium, we foster an inclusive, supportive, fun yet challenging team environment. We  value having a team that is made up of a diverse set of backgrounds and respect the healthy expression of diverse opinions. We embrace experimentation and the examination of all kinds of ideas through reasoning and testing. Come join us as we continue to change the world of digital media. Medium is an equal opportunity employer.

Please mention that you come from Remotive when applying for this job.

Help us maintain Remotive! If this link is broken, please just click to report dead link!

similar jobs

  • Heroku / Salesforce (North America)
    3 days ago

    Heroku is looking for a Developer Advocate with past experience in a developer-facing role, creating demos, writing blog posts, and building communities.

    Heroku’s Developer Advocacy team is small but scrappy and distributed around the U.S. We educate, inspire, and spark community among developers and also advocate internally for them.

    Do you consider yourself a software developer, but you are also excited about developer communities and helping other developers? If that sounds interesting to you, please read on.

    Heroku is looking for a Developer Advocate with past experience in a developer-facing role, creating demos, writing blog posts, and building communities. This role can be remote or you can work from a Salesforce office, but we would like you to be in a North American timezone. Also, this role will require 10-30% travel.


    > What is Heroku?

    Heroku’s mission is to create the fastest path for application developers to turn an idea into a production app of any scale. We want to remove every obstacle that slows down creative coders and let them focus on their customers’ experience while leaving everything else to our service. Today, Heroku powers over 7 million apps ranging from personal hobby apps to Internet top 100 apps. In aggregate, we serve more than 23 billion requests per day. Yet, it only takes minutes for a new developer to get the first app up and running, ready to serve customers.

    Heroku is one of the fastest growing businesses at Salesforce.

    Heroku’s Developer Advocacy team is small but scrappy and distributed around the U.S. We educate, inspire, and spark community among developers and also advocate internally for them. Sometimes this means working individually on some code or a demo or a conference talk. Other times, this means working closely with other Heroku teams like Product, Marketing, or Engineering.

    > Some Things You Might Work On

    • Interact (virtually and in-person) with developers

    • Write blog posts

    • Organize meetups

    • Create product demos

    • Record podcasts

    • Test new product features

    • Create conference talks

    • Attend events

    • Design and build a community program

    • Collect and share developer trends, desires, and feedback internally

    • Mentor other developers (writing and speaking)

    • Maybe most importantly, decide which of these are important and define how we work on them


    > Who you are


    CODE

    You're a developer first. As a coder, you can empathize with other developers, build credibility speaking to them, and even sit down with them in a debugging session.

    You have experience with at least one of Heroku’s supported languages: Node, Ruby, Java, PHP, Python, or Go and something else. Experience meaning not just an understanding of the syntax but also the community. That “something else” could be data, Apache Kafka, API design, web performance, mobile development, continuous delivery, microservices, distributed systems design/architecture, AI/ML, etc. Share what you're excited about with us.

    But you aren't satisfied calling yourself just a developer.

    COMMUNITY

    You also like engaging deeply with developer communities. Even better, you've helped build or grow a developer community. This means you've participated in or helped to organize meetups, conferences, or hackathons. Or it could mean that you've been an integral member of a popular open source project, or you've otherwise participated in a community in a meaningful way virtually. Ideally, more than one of these things.

    CONTENT

    Content is how the Developer Advocate communicates their ideas and the meaning behind their code. Whether it's writing a technical blog post, scripting out an engaging demo, crafting a clear GitHub repo README, speaking in a podcast, or presenting an idea or project in a conference talk, content allows us to share our message with and hear feedback from a larger audience.

    You should relish the thought of composing clean, clear prose almost as much as you do writing concise, elegant code and not be averse to speaking in front of a group of your peers.


    Other Desirable Characteristics or Background

    • Infectious energy

    • Excited about Heroku’s desire to provide simple, intuitive tools to developers

    • Experience at a company with a developer-focused product

    • Comfortable (as much as one can be) with public speaking

    • Comfortable speaking with customers

  • Jatheon Technologies Inc. (Europe)
    1 week ago

    SENIOR VUE.JS DEVELOPER

    We are a growing company with an internal development team that is working on amazing new projects and looking for a Vue.js Developer to further improve our performance and help us build our Cloud Archiving solution. You will be responsible for maintaining and developing highly-intuitive user interface and will collaborate closely with Design, Application Engineering and QA to specify, build, test and deploy new features. You will not only be a great contributor to the quality of our code, but also have important input on our UX strategy, as well as the opportunity to help us build our technical landscape, participate in developing our design system and share your expertise in front-end development with the rest of the team.


    You will:

    • Define coding standards and best practices for the front end
    • Write well-documented, extensible code that is easy to maintain and that adheres to predefined coding standards
    • Own the overall quality of code including unit testing and functional testing
    • Find and address performance issues
    • Work collaboratively and consult with our development team, provide guidance to the development process and perform code reviews
    • Assist in the architecture, development and integration of software components into fully functional software systems
    • Stay up-to-date on emerging technologies and discuss potential solutions to problems

    Required Skills & Experience:

    • 3+ years of experience in front-end web development
    • Minimum 1 year of extensive experience with Vue.js, alternatively with Angular/React
    • Extensive hands-on experience with JavaScript (closures, scoping, functional concepts, OOP)
    • Knowledge of HTML5 and CSS3 including preprocessors
    • Expert knowledge of jQuery and BootStrap
    • Experience with Single Page Applications
    • Excellent verbal and written communication skills in English
    • Motivation to learn new technologies, continually improve and work with little direction by understanding business goals and performing self-directed research

    Bonus points for experience with:

    • Writing unit tests
    • Local Storage and ES6

    What we offer: 

    • Full-time remote position
    • Flexible hours ‒ it’s about getting the work done, not about when desk chairs are filled
    • High-end laptop and other equipment
    • Great working atmosphere with regular team building activities
    • A chance to be part of a casual, but highly professional international team
    • Conferences, trainings and events
    • Competitive compensation depending on experience and skills
    • Exposure to emerging technologies and practices (Cloud, AWS...)


    If you think we'd be a good fit, please send us a cover letter and resume in English.

  • Jilt (Worldwide)
    1 week ago

    Howdy 👋We’re the folks at Jilt, an email marketing platform built for eCommerce. We help thousands of stores grow sales with emails that delight their customers. Our team is 100% remote, with 21 people distributed all over the world. 

    We’re looking for a smart, detail-oriented front-end engineer to join our team and help build fast, intuitive interfaces in React.

    You should be...

    • Deeply experienced with React, JavaScript, and CSS
    • Adept at translating user stories into detailed engineering implementation tasks
    • Detail-oriented & curious, with a drive to understand and solve technical challenges
    • A self-starter, comfortable working independently and taking a high level of responsibility
    • Dedicated to constant learning and sharing your knowledge

    Bonus points if you also have…

    • Experience using React alongside Ruby on Rails, or other MVC frameworks
    • Angular/Vue experience

    You’ll be responsible for…

    • Building features, tweaks, bug fixes, and refactors based on specs in Clubhouse stories (derived from both customer feedback and our product/support teams)
    • Adhering to internal code standards and best practices while suggesting new ones for team consideration
    • Carefully user-testing your own work and implementing feedback from code review
    • Providing detailed code reviews to your teammates
    • Updating our internal wiki & documentation
    • Assisting our support team with troubleshooting challenging issues

    Why you’ll love working with us…

    • Kind, wonderful teammates that enjoy their work as much as you do
    • Competitive salary
    • Annual company retreats (read about last year's adventure in Lake Tahoe!)
    • Fantastic benefits: 27 days paid time-off, full health insurance (medical, dental, and vision), retirement contributions, paid parental leave, and more!
    • Transparent culture (check out our team wiki)

    Curious to hear more? Learn more about the position and apply here. Applications accepted through June 7th. We can’t wait to meet you!

    We support workplace diversity and do not discriminate on any protected class. We believe when we work together as a team of different views, experiences, and ideas, we can build amazing things.

Remotive can help!

Not sure how to apply properly to this job? Watch our live webinar « 3 Mistakes to Avoid When Looking For A Remote Startup Job (And What To Do Instead) ».

Interested to chat with Remote workers? Join our community!