Data Engineering Swiss Knife
Mennyi a nettója?
Évi 1500 dollár konferencia keret + Medicover + angolórák + hardver és szoftvervásárlási keret
About our client
The company currently employs around 40 people in total, 7 in Budapest, 2 in Barcelona and the rest in San Francisco. It is privately owned. The Budapest team was created by acquiring a small tech firm with a similar product. The aim is to double the number of employees and scale up operations within one year and focus more on Enterprise clients.
Teams have lots of autonomy and room to play with ideas and experiment, as innovation is part of the company’s identity. They are flexible and demand flexibility, i.e. startup mindset with lots of conflicts and unworked-out practices. The company values diversity and expects all members to do the same. It is competing in an industry which is very new. This means there will be lots of changes and it is impossible to know what is going to be the best practice or industry standard in the future.
They appreciate good working relationships and environment, thus, there are many team events, such as Presentation Club, Book Club, board games, sports and a biweekly dinner together. The entire team is travelling to San Francisco twice a year for face-to-face meetings and bonding.
About the product
The company offers a coding error monitoring tool which frees up significant time and thus enables software developer teams (clients) to focus on their product development. The solution is giving developers intelligent information about their code and coding habits. Over 100,000 developers use the product, including some of the best engineering teams in the world.
The service monitors code errors and provides real-time alerting. It is integrated into the engineering environment of the clients and has a dashboard to visualize information and help debugging. It is also integrated with other DevOps monitoring, alerting and ticketing systems to help the developer workflow. The aim is to incorporate ever more intelligent solutions by monitoring thousands of errors in different code bases and seek patterns by using AI and ML tools. The product is a self-service offering, however there is a support team to assist clients.
The technology stack is:
- Frontend: React, Webpack, Sass
- Backend: Python, Node.js, Scala
- Database: MySQL, Elasticsearch, Redis, Memcached, Spark
- Infrastructure: Google Cloud Platform, Kubernetes, Kafka, Terraform, Ansible, Consul, CircleCI, Rollbar
- Support: Intercom, AskNicely, Clubhouse, Slack, Periscope (Sisense), Stripe, Github, Mailgun
It is a cloud native, SaaS product, getting transformed to microservices.
About the team
A new team called “Intelligence” is being created with the supervision and lead of the founder of the acquired Hungarian company. This team will be very important as the company is becoming ever more data-driven. The new colleague has to be able to challenge assumptions, and – especially in the first period – behave as a Swiss knife, being a generalist, who is knowledgeable in various aspects of data analysis, mining, science and infrastructure. It is later to be seen, which member of this new team chooses which sub-task within the range of options, so for the moment, you have to be flexible.
- Conceptualize and generate infrastructure that allows big data (100 million+ data points per day) to be analysed and processed effectively, so that useful and various insights can be given to users real time. You do not have to actually build the infrastructure or platforms, that is for the infrastructure team in S.F.
- Test structures to ensure that they are fit for use. Ask the right questions to help improve the information architecture of the product. The current version is not particularly optimized for such data-driven analysis, so engineering must be supported in transforming the product in a way, that allows users to do analysis themselves and get the right tools for it.
- Transform raw data for manipulation by Data Scientists, improve the data processing pipeline.
- Automate existing and developed database routines and give advice on optimizations across the data activity spectrum.
- Provide different solutions and draw up pros and cons for decision makers, indicate trade-offs, ensure the “bleeding edge” is balanced with the “tried and tested”. (Buzzword only solutions will not be useful.)
- Ask relevant questions about the existing release of the product and its workings, analyse the precise nature and content of the user data collected to see opportunities or determine missing or suspicious parts.
- Work together with the Product Manager and the engineering and customer support teams.
- Help build the Intelligence Team, which would focus on the data analysis tasks of the company.
- Experience in modern big data infrastructure and tools.
- Cloud solutions (GC, AWS etc.)
- Python (Scala or other functional languages are an advantage).
- SQL, MySQL, Elasticsearch, Redis, Memcached, (Analytics databases, Hadoop, AWS S3, Cassandra, Kafka, MongoDB etc. are an advantage too).
- Tensorflow/Pytorch or other ML tools, Spark, Kafka, Akka are a plus.
- Fluent English is a must.
- MSc in Computer Science is preferable.
- Interest in AI, ML, Neural networks, etc. is a great plus.
- You are flexible, open and a quick learner. More of an initiator than an implementer. You are curious, think about business and users, and not just a technology enthusiast. You have a problem-solver attitude, you take your own initiative and do not require to be micromanaged.
Due to the current situation, all interviews will be conducted online, via video conferencing tools. If these are successful, reference checks will be conducted, so one ex-manager and one ex-team member’s phone availability should be provided.