This post was originally published on here
Who is a data scientist? What does he do? What steps are involved in executing an end-to-end data science project? What roles are available in the industry? Will I need to be a good coder to be a good data scientist? How do we solve a data science problem in healthcare without domain knowledge of healthcare? Hold your horses. We will answer all the questions this week.
Data scientist
Creating a data science solution involves the following eleven steps: 1) Define the business problem, 2) Convert the business problem into an analytics problem, 3) Identify tables and columns relevant to the problem at hand, 4) Collect data, 5) Prepare data, 6) Explore data and derive insights, 7) Train model, 8) Evaluate model, 9) Deploy model, 10) Monitor and retrain model, and 11) Retire model. In a true sense, a data scientist should be able to do all the above. However, you may wonder whether a single individual could be an expert in every step of the process. You are right. That’s why we consider a “team” possessing the full range of skills rather than an individual. At the same time, one should not claim to know only how to evaluate models. That will be too narrow a skillset.
Machine learning expert
Machine learning experts typically train models, evaluate them, and select the best-performing models. They prepare data through cleaning, preprocessing, and feature engineering to ensure quality inputs for models. Based on the use case, they choose a modelling technique. Based on the selected method, they evaluate various models using metrics. There are different model evaluation metrics for different modelling techniques, such as regression and classification. Machine learning experts are typically proficient in statistics. In the generative AI era, a machine learning expert should be able to use or fine-tune large language models.
AI engineer
Machine learning engineering involves building applications that leverage traditional machine learning models. AI engineering is building applications to leverage large language models (LLMs).
Visualisation expert
Visualisation experts excel in converting complex technical or textual information into easily understandable representations. The extent of simplification depends on the audience. For your Chief Technology Officer (CTO), who is technically aware and strong, specific technical details may stay. For your business stakeholders, technical information must make way for business solutions, impact, and solution usage details. A visualisation expert in data science transforms complex datasets into intuitive graphical representations, such as charts, dashboards, and interactive visualisations, to uncover insights and patterns. The use of Python libraries such as Matplotlib and Seaborn, and tools such as Tableau, Power BI, and QlikView, is common.
Business analyst
A Business Analyst serves as the bridge and interface between business stakeholders and the data science team. They must be adept at understanding business as well as machine learning and solution development techniques. This is not very different from the typical business analysts we engage with for non-data science projects.
Programmer
A data science team must have a programmer. The data science solution must be integrated with the existing business processes and applications. At times, the solution could be a standalone application. Only then can the machine learning models be consumable by the users. Enter full-stack development: frontend, API, application, business logic, database, infrastructure, and hosting.
ML researcher
When we use the linear regression algorithm, we don’t think much about whether the algorithm’s internal workings are correct. We assume it is accurate and continue to use it to build linear regression models to solve business problems. Not if you are a machine learning researcher. Researchers design and refine machine learning algorithms.
Domain expert
A domain expert is quite essential in the team. You cannot solve a business problem in manufacturing without understanding all that happens in manufacturing. Usually, the business analyst covers us here. However, in practice, the business analyst ends up being more of a technical specialist than a domain specialist, and hence the need.
What do you want to become? “None of the above” is a nice answer too, because AI isn’t everything.
Disclaimer
Views expressed above are the author’s own.
