Software development

As part of our Osedea Portraits series, meet Isabelle Bouchard, senior developer and machine learning specialist. Driven by an insatiable curiosity, Isabelle passionately explores the world of AI and its applications for the well-being of all. Beyond her sharp expertise, discover a sparkling and inspiring woman who will share with us her passions and the many facets of her personality.

Where does your interest in science and artificial intelligence come from?

It was a bit by chance (or luck!) that I decided to pursue a career in artificial intelligence. Initially, I studied biomedical engineering. At the end of my studies, although I had an interest, I had very little experience in software development, and even less in artificial intelligence. In truth, I'd never even heard of the concept of machine learning (ML) until I explored its techniques in my first job.

I immediately liked the fact that it required both mathematical and software development skills, as well as a good ability to break down a problem. I did a bit of self-study, supported by colleagues who helped me progress a lot. I then decided to do a master's degree to formalize my learning and develop research experience.

What are you most passionate about in AI?

I particularly appreciate the dynamism of this field, which evolves very quickly. I am constantly challenged and I have to learn continuously to be able to stay up to date with the technologies.

Moreover, the applications of AI are endless, which allows me to explore new application domains. In recent years, I have worked on projects in health, agriculture, urban planning, and many other areas. It is extremely stimulating.

In cutting-edge sectors such as artificial intelligence, women represent only 22% of professionals. In addition, women represent only 28% of engineering graduates and 40% of computer science graduates. What do you think explains the low number of women in technology, engineering, and artificial intelligence?

There are many reasons why women choose fields other than technology, but it is clear to me that the lack of female role models is one that weighs heavily in the balance. This also explains why many women choose to leave the field, where they do not always feel at home. We naturally develop affinities with people who resemble us, so it can sometimes be more difficult to find allies or mentors when you are a woman in an environment that is predominantly male. We also have a fairly well-defined preconceived idea of the typical developer, and it can be scary when you don't identify with it. Personally, it affected me for a long time, but I learned over time to recognize that, clearly, I will never fit that profile, but that it is more often seen as a strength than a weakness.

How do you see the impact of AI and machine learning on industries in the years to come, especially in areas such as healthcare, finance, and technology?

I hope to see AI assist humans, rather than replace them, especially in healthcare. Our systems are increasingly lacking in humanity, and I hope that our leaders will be able to understand how to use technologies to make us more efficient, but without losing sight of the importance of human relationships. You will understand that I am not necessarily the optimistic type! AI and technology in general offer endless possibilities, but we still need to know how to use them wisely.

How does the AI team at Osedea integrate ethical principles into the design and deployment of AI-based solutions?

The ethics of AI projects still largely depends on the people who develop them. So I would spontaneously say that it all starts with hiring people who have these issues at heart. Then, in our process of analyzing the needs of a project, we have several checkpoints to ensure that we raise and mitigate the main ethical risks. For example, we analyze the datasets used to train our AI models to ensure that we do not favor one segment of users over an underrepresented group in the data.

What would be your ideal retirement?

A healthy one! I love the outdoors. I would like to have a retirement filled with adventures that would allow me to continue to push myself physically and mentally, and to marvel at nature. All this accompanied by my favorite adventure partner, our family and friends... and of course, interspersed with good moments of relaxation!

Do you have a favorite movie or book to recommend, and why?

Fresh from the Oscars, I can't recommend the sublime film "Poor Things" highly enough, starring the incredible Emma Stone. Like all of Yorgos Lanthimos' films, it stands out for its intricate screenplay and utterly unsettling characters. A true gem!

I also highly recommend reading the classic American novel "The Grapes of Wrath" by John Steinbeck. Published in 1939, this novel tells with great accuracy and enduring relevance the human experience of those who lose the life lottery and find themselves in poverty. A deeply moving story.

What is your best and worst habit?

My best habit is to be active. It's a constant effort, but it's so beneficial to me that I make it a daily priority. My worst habit...procrastinating, without a doubt, but I'm working on it!

What's your best trip and why?

It was a trip to Turkey, in 2015, when I had just finished my university degree. I loved the rich culture, the grandiose landscapes and the absolutely vibrant cities, but above all, the incredible sense of freedom I was lucky enough to experience while traveling in my early twenties. With no money and no time constraints, we really let ourselves be carried along by the encounters and opportunities that came our way. I know I'll never be able to do that again - it was magical!

If you could instantly possess one skill, what would it be and how would you use it in your daily life?

I would love to have the ability to speak fluently and confidently on the spot, in a clear and concise manner. I truly admire people who can do this naturally! I would apply this skill in all areas of my life, both personally and professionally. Communication is so important in all of the relationships we have. It would be magical to be able to do this in multiple languages!

In this article, we'll look into the critical importance of DataOps and MLOps in software and AI development. We will showcase a hands-on MVP approach, emphasizing the use of DVC (Data Version Control) and CML (Continuous Machine Learning), integrated with Git, to illustrate these concepts effectively.

• Practical Approach: Using DVC and CML, we demonstrate a real-world, minimal viable product (MVP) approach to DataOps and MLOps.
• Integration with Git: Highlighting the seamless integration of these tools with Git, we show how familiar workflows can be enhanced for data and model management.
• Effective Implementation: Our goal is to provide clear guidance on effectively implementing DataOps and MLOps practices.

Common Issues in AI & Data Projects

1. "Which Data Version?"Are you constantly losing track of the data version used for model training?
2. "Is the New Model Any Good?"Stop wondering if your latest model beats the old one or what changed between them.
3. "Why's Our Repo So Heavy?" Bloated GitHub repository with data?

What is Understanding DataOps and MLOps

DataOps and MLOps are foundational practices for modern software development, particularly in AI. These approaches are essential for effectively managing the data and machine learning model lifecycles.

• Scalability: Efficiently managing data (DataOps) and machine learning models (MLOps) is critical to building scalable and robust AI systems, crucial for software development projects.
• Performance and Reliability: Implementing these practices ensures consistent system performance and reliability, which is especially vital for startups operating in dynamic and resource-constrained environments.
• Pitfalls to Avoid: Many development teams need to correctly version data and models or take a reactive approach to system management, leading to significant challenges in reproducibility and increased error rates, hindering growth and innovation.

Understanding and integrating DataOps and MLOps into workflows is not just beneficial; it's a strategic necessity.

The MVP Approach

The MVP (Minimal Viable Product) approach in DataOps and MLOps is all about aligning with the core principles of the Agile Manifesto, emphasizing simplicity, effectiveness, and deployment.

• Agile Principles: Emphasize simplicity, effectiveness, and people-first processes, promoting flexibility and responsiveness in project management.
• Reducing Dependency on Complex Systems: Advocate for minimizing reliance on complex SaaS and proprietary systems, thus maintaining control and flexibility in your development.
• Effective Tools: Leverage tools like DVC and CML that integrate with familiar Git workflows; this approach ensures seamless adoption and enhances team collaboration and efficiency.

Adopting an MVP approach means creating more agile, adaptable, and efficient workflows in DataOps and MLOps, allowing for the development of robust and scalable solutions without getting bogged down by unnecessary complexities.

Hands-On

Now, we dive into the practical aspects of setting up a Python environment and using essential tools like DVC, CML, and SciKit-Learn. We'll go through configuring a GitHub repository for effective version control and demonstrate building and evaluating a model using SciKit-Learn in a Jupyter Notebook.

• Setup: Set a Python environment and install DVC, CML, and SciKit-Learn.
• Model Building: Use SciKit-Learn with a built-in dataset in a Jupyter Notebook for a simple model training and evaluation demonstration.
• Streamlined Process: Configure GitHub and Git to execute and assess your model.

Install Python Environment

We'll use Poetry to manage our Python environment. Poetry is a Python dependency management tool that allows you to create reproducible environments and easily install packages.

# Install Poetry
pipx install poetry

# Init Poetry project
poetry init

# Add dependencies
poetry add dvc cml scikit-learn

Loading the Data

We'll use the Breast Cancer Data Set from the UCI Machine Learning Repository.

Key characteristics:

• Number of Instances: 569.
• Number of Attributes: 30 numeric, predictive attributes, plus the class.
• Attributes: Measurements like radius, texture, perimeter, area, smoothness, compactness, concavity, concave points, symmetry, and fractal dimension.
• Class Distribution: 212 Malignant, 357 Benign.

import sklearn.datasets

# Load dataset
data = sklearn.datasets.load_breast_cancer(as_frame=True)
print(data.data.info())

Implementing External Settings for Data and Model Adjustments

We'll use external configuration files, like settings.toml, to dynamically adjust data and model parameters. This approach adds flexibility to our project and underscores the importance of versioning and tracking changes, especially when introducing intentional alterations or "bugs" for demonstration purposes.

Degrading the Data with External Settings

Because the demonstration dataset performs well with a simple model, we'll artificially degrade the data to emphasize the importance of tracking changes and versioning.

• External Configuration: Utilize settings.toml to set parameters like num_features=1, which dictates the number of features to be used from the dataset.
• Data Manipulation: We dynamically alter our data by reading the num_features setting from settings.toml. For instance, reducing the dataset to only one feature:

python
import toml

settings = toml.load("settings.toml")
data.data = data.data.iloc[:, : settings["num_features"]]
print(data.data.info())

Training the Model

We'll use SciKit-Learn to split the data and train a simple model.

python
import sklearn.model_selection

# Split into train and test
X_train, X_test, y_train, y_test = sklearn.model_selection.train_test_split(
data.data, data.target, test_size=0.3, random_state=42
)

python
import sklearn.linear_model

# Train a simple logistic regression model
model = sklearn.linear_model.LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

python
# Evaluate the model
predictions = model.predict(X_test)
accuracy = sklearn.metrics.accuracy_score(y_test, predictions)
print(f"Model Accuracy: {accuracy:.2f}")

Model Accuracy: 0.91

python
# View the classification report
report = sklearn.metrics.classification_report(y_test, predictions)
print(report)

# Export the report to a file
with open("report.txt", "w") as f:
f.write(report)

                 precision    recall  f1-score   support
 
              0       0.93      0.83      0.87        63
              1       0.90      0.96      0.93       108
 
       accuracy                           0.91       171
      macro avg       0.92      0.89      0.90       171
   weighted avg       0.91      0.91      0.91       171

python
import seaborn as sns
import matplotlib.pyplot as plt

# Create a confusion matrix
confusion_matrix = sklearn.metrics.confusion_matrix(y_test, predictions)

# Plot the confusion matrix
sns.heatmap(confusion_matrix, annot=True, fmt="d")

# Export the plot to a file
plt.savefig("confusion_matrix.png")

‍

‍

Saving the Model and Data

We'll save the model and data locally to demonstrate DVC's tracking capabilities.

python
from pathlib import Path

# Save data
Path("data").mkdir(exist_ok=True)
data.data.to_csv("data/data.csv", index=False)
data.target.to_csv("data/target.csv", index=False)

python
import joblib

# Save model
Path("model").mkdir(exist_ok=True)
joblib.dump(model, "model/model.joblib")

Implementing Data and Model Versioning with DVC

Until now, we have covered the standard aspects of AI and machine learning development. We're now entering the territory of data versioning and model tracking. This is where the real magic of efficient AI development comes into play, transforming how we manage and evolve our machine-learning projects.

• Better Operations: Data versioning and model tracking are crucial for AI project management.
• Data Versioning: Efficiently manage data changes and maintain historical accuracy for model consistency and reproducibility.
• Model Tracking: Start tracking model iterations, identify improvements, and ensure progressive development.

Streamlining Workflow with DVC Commands

To effectively integrate Data Version Control (DVC) into your workflow, we break down the process into distinct steps, ensuring a smooth and understandable approach to data and model versioning.

Initializing DVC

Start by setting up DVC in your project directory. This initialization lays the groundwork for subsequent data versioning and tracking.

dvc init

Setting Up Remote Storage

Configure remote storage for DVC. This storage will host your versioned data and models, ensuring they are safely stored and accessible.

dvc remote add -d myremote /tmp/myremote

Versioning Data with DVC

Add your project data to DVC. This step versions your data, enabling you to track changes and revert if necessary.

dvc add data

Versioning Models with DVC

Similarly, add your ML models to DVC. This ensures your models are also versioned and changes are tracked.

dvc add model

Committing Changes to Git

After adding data and models to DVC, commit these changes to Git. This step links your DVC versioning with Git's version control system.

git add data.dvc model.dvc .gitignore
git commit -m "Add data and model"

Pushing to Remote Storage

Finally, push your versioned data and models to the configured remote storage. This secures your data and makes it accessible for collaboration or backup purposes.

dvc push

Tagging a Version

Create a tag in Git for the current version of your data:

git tag -a v1.0 -m "Version 1.0 of data"

Updating and Versioning Data

1. Make Changes to Your Data:
   
   -Modify your data.csv as needed.
2. Track Changes with DVC:
   
   -Run dvc add again to track changes:

dvc add data

Commit the New Version to Git:

-Commit the updated DVC file to Git:

git add data.dvc
git commit -m "Update data to version 2.0"

Tag the New Version:

-Create a new tag for the updated version:

git tag -a v2.0 -m "Version 2.0 of data"

Switching Between Versions

Checkout a Previous Version:

-To revert to a previous version of your data, use Git to checkout the corresponding tag:

git checkout v1.0

Revert Data with DVC:

-After checking out the tag in Git, use DVC to revert the data:

dvc checkout

Understanding Data Tracking with DVC

DVC offers a sophisticated approach to data management by tracking pointers and hashes to data rather than the data itself. This methodology is particularly significant in the context of Git, a system not designed to efficiently handle large files or binary data.

How DVC Tracks Data

Storing Pointers in Git:

-DVC stores small .dvc files in Git. These pointers reference the actual data files.

-Each pointer contains metadata about the data file, including a hash value uniquely identifying the data version.

Hash Values for Data Integrity:

-DVC generates a unique hash for each data file version. This hash ensures the integrity and consistency of the data version being tracked.

-Any change in the data results in a new hash, making it easy to detect modifications.

Separating Data from Code:

-Unlike Git, which tracks and stores every version of each file, DVC keeps the actual data separately in remote storage (like S3, GCS, or a local file system).

-This separation of data and code prevents bloating the Git repository with large data files.

Importance in the Context of Git

Efficiency with Large Data:

-Git struggles with large files, leading to slow performance and repository bloat. DVC circumvents this by offloading data storage.

-Developers can use Git as intended – for source code – while DVC manages the data.

Enhanced Version Control:

-DVC extends Git's version control capabilities to large data files without taxing Git's infrastructure.

-Teams can track changes in data with the same granularity and simplicity as they track changes in source code.

Collaboration and Reproducibility:

-DVC facilitates collaboration by allowing team members to share data easily and reliably through remote storage.

-Reproducibility is enhanced as DVC ensures the correct alignment of data and code versions, which is crucial in data science and machine learning projects.

Using DVC as a Feature Store

DVC can be a feature store in machine learning workflows. It offers advantages such as version control, reproducibility, and collaboration, streamlining the management of features across multiple projects.

What is a Feature Store?

A feature store is a centralized repository for storing and managing features - reusable pieces of logic that transform raw data into formats suitable for machine learning models. The core benefits of a feature store include:

1. Consistency: Ensures uniform feature calculation across different models and projects.
2. Efficiency: Reduces redundant computation by reusing features.
3. Collaboration: Facilitates sharing and discovering features among data science teams.
4. Quality and Compliance: Maintains a single source of truth for features, enhancing data quality and aiding in compliance with data regulations.

Benefits of DVC in Feature Management

• Version Control for Features: DVC enables version control for features, allowing tracking of feature evolution.
• Reproducibility: Ensures each model training is traceable to the exact feature set used.
• Collaboration: Facilitates feature-sharing across teams, ensuring consistency and reducing redundancy.

Setting Up DVC as a Feature Store

1. Organizing Feature Data: Store feature data in structured directories within your project repository.
2. Tracking Features with DVC: Use DVC to add and track feature files (e.g., dvc add data/features.csv).
3. Committing Feature Changes: Commit changes to Git alongside .dvc files to maintain feature evolution history.

Using DVC for Feature Updates and Rollbacks

• Updating Features: Track changes by rerunning dvc add on updated features.
• Rollbacks: Use dvc checkout to revert to specific feature versions.

Best Practices for Using DVC as a Feature Store

• Regular Updates: Keep the feature store up-to-date with regular commits.
• Documentation: Document each feature set, detailing source, transformation, and usage.
• Integration with CI/CD Pipelines: Automate feature testing and model deployment using CI/CD pipelines integrated with DVC.

Implementing a DVC-Based Feature Store Across Multiple Projects

1. Centralized Data Storage: Choose shared storage that is accessible by all projects and configure it as a DVC remote.
2. Versioning and Sharing Features: Version control feature datasets in DVC and push them to centralized storage. Share .dvc files across projects.
3. Pulling Features in Different Projects: Clone repositories and pull specific feature files using DVC, enabling their integration into various workflows.

Best Practices for Managing a DVC-Based Feature Store Across Projects

• Documentation: Maintain comprehensive documentation for each feature.
• Access Control: Implement mechanisms to regulate access to sensitive features.
• Versioning Strategy: Develop a clear strategy for feature versioning.
• Automate Updates: Utilize CI/CD pipelines for updating and validating features.

Streamlining ML Workflows with CML IntegrationIntegrating Continuous Machine Learning (CML) is a game-changer for CI/CD in machine learning. It automates critical processes and ensures a more streamlined and efficient workflow.Setting Up CML WorkflowsCreate a GH Actions workflow within your GitHub repository, ensuring it is configured to run on every push or PR.name: model-training
on: [push]
jobs:
 run:
runs-on: ubuntu-latest
steps:
  - uses: actions/checkout@v4
  - name: Install Poetry
    run: pipx install poetry
  - name: Set up Python
    uses: actions/setup-python@v4
    with:
      python-version: "3.10"
      cache: "poetry"
  - name: Install dependencies
    run: poetry install --no-root
  - uses: iterative/setup-cml@v2
  - name: Train model
    run: |
      make run
  - name: Create CML report
    env:
      REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    run: |
      echo "\`\`\`" > report.md
      cat report.txt >> report.md
      echo "\`\`\`" >> report.md
      echo "![](./confusion_matrix.png)" >> report.md
      cml comment create report.md

Conclusion: Boosting Software and AI Ops

In wrapping up, we've delved into the core of DataOps and MLOps, demonstrating their vital role in modern software development, especially in AI. By mastering these practices and tools like DVC and CML, you're learning new techniques and boosting your skillset as a software developer.

• Stay Agile and Scalable: Adopting DataOps and MLOps is essential for developing in the fast-paced world of AI and keeping your projects agile and scalable.
• Leverage Powerful Tools: Mastery of DVC and CML enables you to manage data and models efficiently, making you a more competent and versatile developer.
• Continuous Learning and Application: The journey doesn’t end here. The true potential is realized in continuously applying and refining these practices in your projects.

This is more than just process improvement; it's about enhancing your development workflows to meet the evolving demands of AI and software engineering.

‍

‍

At Osedea, we have a very unique organizational structure. We’re inspired by humanocracy, an approach coined by Gary Hamel —a researcher at Harvard University. His data-driven argument is about making efforts to bypass bureaucracy, forgo managers, and shift an organization’s DNA to give team members an equal chance to learn, grow, and contribute.

This is a new experience for many people in the workplace, so we wanted to share our thoughts and answer any questions you might have about joining a company like ours that operates with this kind of structure.

First things first, what is a bureaucracy?

Bureaucracies originated in the 18th century. They were designed around position and individuals to address nepotism (assigning positions of power by way of favouritism to friends and family members). Bureaucracy was also designed to have rules, practices, and principles to maximize compliance of a group of people. Later in the 20th century, sociologist Max Weber remarked that, “bureaucracy is more perfect when it’s more dehumanizing.”

You’ve likely worked in a bureaucracy, as the vast majority of organizations are structured this way:

• There’s a formal hierarchy (a top-down approach and many levels)
• Power is vested in positions (people with certain roles have more power than others)
• Managers assign tasks and assess performance
• Everyone competes for promotions and compensation correlates with rank

Gary Hamel’s research showed some concerning data about bureaucracies:

• 79% of people polled said bureaucracy significantly slows decision-making
• 68% said, in their organization, new ideas are met with skepticism or outright resistance
• 76% said political behaviours highly influence who gets ahead, not competence or potential

Maybe you've even been frustrated in a traditional hierarchical environment? Here’s why organizations stick with it:

• They’re a familiar way to organize humans into action
• It’s hard to imagine alternatives
• They work to a certain extent; things get done, we’re able to control/coordinate, and to have consistency
• As bureaucracies have been around since the 18th century, millions of careers have been built around the desire to climb corporate ladders and attain positions of power; we’re reluctant to change as humans

Letting go of the familiarity of bureaucracy requires courage, creativity and a desire for a more humane organization. As a society we can work towards this.

From bureaucracy to humanocracy

Osedea was structured around the individuals within our organization, as opposed to a corporate structure. In our opinion, bureaucracies can be dehumanizing. They can encourage bad behaviour in people, power plays, and politics. Those who get ahead aren’t necessarily rewarded for behaviours that are helpful to the organization. Senior management tends to reward people who prioritize their self-interests, those who excel at “managing up,” and people who are good at reading management moods/meeting their manager’s needs (as opposed to the organization’s needs).

At Osedea, we decided on a “structure without managers.” This is a bit different than a “flat structure,” which is a widely-, but oftentimes incorrectly-, used term to describe a structure with only a few layers of management and a short chain of command, that can hardly survive growth and appears to be the dream but ends up being a logistical nightmare and chaos. Interested about learning more, here are you a few things that you might not know about a flat structure.

Why we chose a structure without managers

• This type of structure is in line with our vision of creating a world of opportunities for our team while making their work life enjoyable.
• It boosts creativity, learning and autonomy.
• It fosters “horizontal” instead of “vertical” ambition. "Vertical" ambition is the usual career-path trajectory, in which a newbie moves up the ladder from associate to manager to vice president over a number of years of service. "Horizontal" ambition is where employees who love what they do are encouraged to dig deeper, expand their knowledge, and become better at it. This doesn’t mean horizontal has no evolution. Instead of rewarding high performers with managerial responsibilities—which often drives people further away from the job they are actually good at—we reward them with responsibilities closer to the work. We complement that with values, benefits and autonomy.
• People can make contributions at Osedea no matter what their experience level is.
• It drives results faster.
• It removes the negative impacts of bureaucracy.

This is not to say that everything in our structure works perfectly—we have to remember that organizations are always a work in progress. We have a destination but there will be challenges. Some will be easy to fix, some will stick around, and new challenges will pop up. It’s sometimes hard to imagine how an institution could function without a formal organization, but we should try.

Some common misconceptions about our structure

Without managers there are no leaders.

It’s important to understand that a manager isn’t necessarily a leader. At Osedea we don’t believe in assigned leadership, we believe in leadership that is built through high quality work, curiosity in building towards our company vision and natural influence with others. There’s always a “lead” on each project or initiative team—someone who is responsible for holding everyone accountable to the delivery of the project and who is the go-to. But, we don’t believe in a structure where there’s a person who is a boss of devs, a person who is the boss of designers, a person who is a boss of sales, etc.

Without managers there is chaos and decisions are impossible to make.

We organize everything through efficient processes, team expectations and natural leadership. You might think that without a boss who is the final decision maker, we can’t move forward, or that everyone gets a say in every decision. People who have ownership over a specific project and full understanding and information of context should contribute. But in the end, someone makes a decision based on information, input and what’s best for our company vision, values, stakeholders and business needs.

Without a manager I can’t have a career progression and there is no mentorship, coaching, or feedback.

We believe in a career of achievements and that there are several paths that lead to success, not only the management path. And even though we don’t have managers, we have processes and metrics in place to ensure that our people grow to their full potential and get proper feedback, that performance is evaluated, and team members get coaching.

Because I can’t get promoted, there are no incentives for me to perform and contribute.

At Osedea, strong performance is rewarded financially and with additional growth opportunities (e.g. when we send our developers to speak at international conferences, even if they aren’t the most senior in our company, or how our full stack developer was given a chance to work 1:1 with Spot the robot). We’re agile, we think outside the box, we reward paths that aren’t on a traditional career ladder, and we ask team members to contribute where they’re strongest and reap the most enjoyment.

Your current structure is fine now that you’re under 100 employees, but there’s no way you can maintain it as you grow.

Growth is key for every company, but not for us if it risks bringing down our humanocracy approach. Can a structure like ours survive long term: YES! Does it mean there won’t be adjustments and improvements? NO. We have to constantly work on it. We stick to our vision, but it doesn’t mean our day-to-day will always be the same.

Today, we have offices in Montréal, Canada and Nantes, France, with another opening soon in the UK. Our international offices are independent but united around the same mission, values and strategies. This way we can ensure that, as we scale up, we maintain our humane approach.

At Osedea, we’re constantly at the forefront of emerging technologies, and we have a unique perspective on tech adoption trends among our diverse client base. In recent months, AI has made its way into mainstream media with the help of ChatGPT. Since then, tooling and support for AI development have skyrocketed. Just a few weeks ago, Dr. Andrew Ng, a globally recognized leader in AI, delivered a talk on the opportunities in AI which highlighted the importance of integrating AI into your organization's workflow.

BEARING.ai—the first company to harness the power of Generative AI in the maritime shipping industry—is a great example of how AI adoption can quickly reap tremendous benefits. Leveraging their data to monitor, forecast, simulate, and optimize, BEARING.ai’s clients have achieved substantial improvements in shipping vessel performance while simultaneously reducing fuel costs and carbon emissions, contributing to a greener environment. Similar opportunities aren't distant dreams; they’re within grasp. AI is ripe for adoption, and the key to unlocking its full potential lies in harnessing your organization's data.

The benefits of centralizing data

For many established companies, data has been accumulating for years across various departments and systems. PDFs, images, presentations, emails, audio, video, and analytics are treasure troves of information (and significant assets when harnessed correctly). The first step towards adopting AI within your organization is centralizing your data. Centralization (consolidating from various sources/locations into a single repository or system) by implementing a unified data management platform or integrating existing systems through middleware solutions offers numerous benefits:

‍

AI-Powered Knowledgebase: Once centralized, data can be organized and indexed efficiently with the help of embedding models. These models are trained to extract the most meaningful information out of your unstructured data. By indexing your data in this manner, Large Language Models such as GPT-4 can have their context extended with your organization's business context to evolve into a comprehensive, all-knowing assistant. This innovative approach is known as retrieval augmented generation (RAG) with vector databases, a concept we will delve into shortly.

Training Predictive Models: The consolidated data pool becomes a valuable resource for training AI models. Predictive analytics, forecasting, and trend analysis become achievable goals as you capitalize on your organization's historical data.

Security Benefits: Centralizing data provides a more robust security infrastructure to safeguard sensitive information. It allows for more effective access control and auditing, reducing the risk of data breaches.

Easier Backups: Centralized data is easier to back up than data from multiple disparate sources. This simplifies data protection measures, ensuring critical information is securely preserved and recoverable in case of data loss incidents.

Redundancy: Implementing redundancy, such as data mirroring or replication, becomes more feasible with centralized data. Redundancy enhances data availability and fault tolerance, minimizing downtime, and ensuring business continuity.

Building an AI-powered knowledgebase

As mentioned above, retrieval augmented generation systems (RAG) have gained prominence as a valuable solution for querying an organization's data using large language models (LLMs). RAG systems allow for querying data with natural language. Essentially, it gives you a way of “talking” to your data in the same way you talk to ChatGPT. The accessibility of LLMs in recent months has made this approach a lot more feasible which is why this approach to data exploration is quickly gaining traction. However, the success of such systems depends not only on LLMs and prompt engineering but also on the proper vectorization and indexing of data. This is where vector databases and embeddings play a crucial role.

‍

What are Vector Embeddings?

In the context of AI and machine learning, vector embeddings are a numeric representation of an entity's semantics. These representations capture essential features and relationships within the data, making it easier for AI algorithms to process and understand. Embeddings are crucial for tasks such as natural language processing, recommendation systems, and image recognition. With embeddings, we can quickly find related content based on similarity. Additionally, embeddings aren’t just limited to text, it’s possible to create vectors out of images, audio, video, or any type of data using encoder models that have been trained to extract their meaningful information. Some models like OpenAIs text-embedding-ada-002 are even language agnostic, meaning that they can understand similarity in various languages natively.

‍

What are vector databases?

A vector database is a specialized database designed to store and retrieve high-dimensional vector embeddings efficiently, making them ideal for AI and machine learning applications. They use approximate nearest neighbor (ANN) search algorithms to measure the distance between embeddings, resulting in a ranked list of neighboring vectors.

As an example, Spotify has been using vector databases for quite some time to compare users' taste in music, they also go into detail on how they’ve used embeddings to query their podcast episodes in their blog post: Introducing Natural Language Search for Podcast Episodes. They’ve even gone as far as creating their own ANN library!

‍

Use cases for vector databases

Vector databases are well-suited for a wide range of use cases that involve similarity search, recommendation systems, and data analysis in fields such as machine learning, natural language processing, computer vision, and more. Here are some common use cases for vector databases:

Recommendation Systems: Vector databases are often used in recommendation systems to find items or content that are similar to what a user has interacted with in the past. This can be applied to e-commerce, content recommendation, and music or video streaming platforms.

Content-Based Search: In multimedia content platforms, vector databases enable content-based search for images, audio, and video files. Users can search for content with similar visual or auditory features.

Anomaly Detection: Detecting anomalies in high-dimensional data, such as network traffic logs, sensor data, or financial transactions, can be done using vector databases. Unusual data points can be identified by comparing them to a set of normal vectors.

Collaborative Filtering: Collaborative filtering algorithms can use vector databases to find users with similar preferences and recommend items based on the behaviour of similar users.

Long-term Memory: Vector databases can be used to store past response generations for an LLM. These embeddings can be recalled to further enhance a large language model’s context with its past context.

Clustering: In vector databases, clustering can be applied to organize data into distinct groups, making it easier to identify patterns and similarities within the dataset.

Diversity Measurement: In vector databases, diversity measurement can be applied to evaluate the breadth and inclusivity of recommendations, ensuring a balanced selection of items or content to cater to a wide range of user preferences or topics.

Pitfalls of embedding models and vector databases

While embeddings are powerful, they are not without their challenges. It's important to be aware of potential pitfalls, such as bias in the training data. As an example, OpenAI explains in their documentation how they’ve witnessed a model more strongly associate European American names with positive sentiment when compared to African American names. Additionally, embedding models have cutoff dates on their training data, meaning some data might differ in semantics throughout time (e.g. a celebrity’s popularity). Selecting the right embedding techniques and parameters is critical to achieving optimal results and correct data pre-processing techniques need to be applied to correctly leverage embeddings.

Vector databases are only half the solution

While vector databases and embeddings are essential components of AI adoption, it's crucial to recognize that they’re part of a more extensive ecosystem. Building a robust AI infrastructure involves addressing other key aspects, such as data preprocessing, model selection, prompt engineering, and deployment strategies. Vector databases are a powerful piece of the puzzle, but they are not the entire solution.

One of the recurring issues with LLMs and AI as a whole is the tradeoff in accuracy. For ages, computers have been binary and deterministic. Although vector databases may be a monumental step forward in knowledge exploration, they still need to be married to traditional structured architectures for the ultimate search experience. Some platforms such as Azure Cognitive Search and Elastic Search are actively working and fine-tuning hybrid searches using reciprocal rank fusion (RRF) to mix resulting ranks. Elastic is also addressing other vector database issues such as privacy of data and role-based access control (RBAC). On the prompt engineering side of things, various frameworks like guidance ai, Langchain, and LMQL are all being developed to provide a robust way of turning LLM data into meaningful structured responses. Needless to say, we are living in exciting times and emerging RAG architectures are only getting better each day.

At Osedea, we recognize the potential of vector databases, embeddings, and AI in driving transformative changes within organizations. Our comprehensive development services are designed to empower your organization in leveraging its data and fully realizing the potential of AI. Whether it's centralizing data, implementing vector databases, or navigating the complexities of AI adoption, Osedea is here to be your trusted partner on the path to data-driven success. Rest assured, we remain at the forefront of emerging technologies and are committed to staying on top of the latest advancements in the field.

‍

Last August, Zoom (famous for helping us connect during the pandemic), ironically recalled their workforce to the office. You’ve likely noticed this trend of companies bringing their teams back to IRL too. I know that, in my network, people are being urged to return to on site work 2-3 times per week.

While I support the argument for boosting engagement and collaboration, I can’t quite understand the decision to impose a set amount of days per week. I anticipate that workers who thrive in remote work environments might become less engaged, or just leave. And will the problem of lower engagement magically be solved, just by having people back in the same physical location? I doubt it. After all, disengagement and lack of cohesion in teams existed pre-pandemic. Perhaps a collective pause is required to bring more creative solutions to the forefront.

In our recent focus groups at Osedea, our team has been clear: they wish their colleagues would be there more in person… but on the flip side of the coin, unilaterally, everyone appreciates the extreme flexibility we offer.

In this piece, I share my reflections and our learnings about shifting work dynamics in hopes that it may help you in your own reflections.

Create memorable moments together

Gregg Popovich, legendary head coach of the San Antonio Spurs takes his team after every game to a great culinary experience. Now, NBA players (many of them millionaires), can certainly afford their own dinner. But, as quoted by former Spurs guard Danny Green in a 2020 ESPN article about Popovich mindset, “dinners help us have a better understanding of each individual person, which brings us closer to each other…and, on the court, understand each other better.”

Indeed, nothing spells team bonding like living memorable and unique experiences together. This is something we were really good at in Osedea’s early days when our team was smaller. Each year, we planned a five-day all-inclusive vacation with our team. The goal was to receive training and bond—boosting engagement and collaboration. You can read more about our 2018 trip to Lisbon here.

As we grew in size, the logistics of arranging regular team trips became more challenging. So this year, we channeled the mindset of our team trips, and applied it to the launch of OsedeaFest: two action-packed days of inspiring training sessions, delicious dining, and exploring our city. The result? A spike in team engagement and connection, as per our results in Officevibe and our team’s feedback.

Organize your in-person days around rituals

There are certain activities that make more sense in person, while others don't. For instance, last summer we resumed in-person interviews. It gives us a chance to connect and make the candidate experience more memorable. Similarly, client demos are a great opportunity to develop rapport and offer a positive collaborative experience. Now, whenever we organize certain types of rituals we ask ourselves: is this better in person, remote, or does either work fine? We now organize our in-person days around this, instead of just assigning an arbitrary number of days people are required to be physically present.

Provide flexibility. Get flexibility in return.

Our team reports loving the flexibility we offer. Some team members like me (thank you, four-minute commute!) come to the office most days. Others are regulars, but without set days. Others still are rarely there in person. However, when we have a day of interviews, we never have to beg our team to show up or negotiate their presence. We know that they respect the process, even if they’re the type of person who prefers remote work.

Communicate differently to engage your teams effectively

One comment that I hear a lot from other company leaders is that they worry that if they go fully remote, their team will be disengaged with their company. If you think being there in person for communication will prevent you from having a disengaged workforce, I invite you to think again. We have to challenge ourselves as leaders to think differently and find innovative ways to communicate and engage the team.

Here are some things we’ve done to enhance communication and engagement:

• Engage your audience by exploring meaningful mediums. Our daily team newsletter ran for three years before it started to lose impact. We shifted gears to a highly engaging medium: an internal podcast. Once a week, we broadcast an episode showcasing different team members. It’s a great way to create a personal connection with audience members and the team can listen whenever it’s convenient for them.
• Revamp existing communication rituals. We had been running our quarterly update the same way (in traditional video conference style with slides and various presenters) since our inception. To make the presentation of this important content more dynamic, we switched things up this fall with a virtual talk show style format.

Final words

According to Indeed, hybrid work is here to stay with more than 42% of posted jobs across industries including hybrid in their offering. But hybrid work can be so much more inspiring and rewarding for your team than a monotonous rule of a set number of days per week at the office. Flexibility and a tailored-to-your-team approach may be stronger differentiators to attract talent and offer rich professional experiences. So, before hopping on the return-to-the-office wagon, I urge you to ask yourself, “can we do better?”

Employees deserve more than the bare minimum dental plan and yoga membership to excel personally and professionally. That’s why, when it comes to taking care of our team members, we make it our responsibility to support them in reaching their life goals. We do this by putting our money where our mouth is to offer high impact, inclusive and equitable benefits.

One area where we’d like to help? Supporting team members who want to start a family, whatever that may look like.

Here are a few reasons why this issue is important to us:

• Québec–home of our founding studio–is ahead of most Canadian provinces in terms of parental benefits, thanks to the Québec Parental Insurance Plan (QPIP). But QPIP still doesn’t cover 100% of a parent’s salary during their leave.
• There are grey areas within Québec’s 2021 Medically Assisted Reproduction Program, which doesn’t account for the realities of many individuals. The program lacks inclusive benefits, especially for surrogates and same-sex couples.
• Due to financial concerns, a common issue for couples starting a family is that one parent may be reluctant to take much parental leave, meaning they lose out on precious time with their child from the very start.

A more equitable parental program

To give our employees the chance to start a family without having to make sacrifices, whatever their gender or sexual orientation, we created our latest benefit program called “Building Families.”

It’s designed to support Osedea families in their parenthood journey, and make that journey inclusive (regardless of gender or diagnosis). This new benefit fills the gaps left by government programs, promotes equality and gives choice, flexibility, and support in the journey to parenthood, which looks different for everyone.

“Deciding to start a family is a big decision, and an exciting one,” says Martin Coulombe, founder of Osedea. “We believe that anyone can be a child’s caregiver, regardless of gender identity or sexual orientation. That’s why we offer everyone at the company equal support to start a family, care for that family, and create a bond with their new child.”

Building Families is one of the first programs in Quebec that provides fertility, adoption and parental-leave perks to help employees build a family their own way. The initiative offers a range of benefits that provide support and flexibility to employees throughout the journey. And while government programs differ between countries, Osedea is offering this policy to employees who work out of any of our three international offices in Montreal, Nantes, and London.

Here are the details of our new program:

• 80% of fertility-treatment costs, up to $20,000 CAD, including:
• • Coverage of extra treatment cycles of ovarian stimulation
  • Coverage of extra treatment cycles of in vitro fertilisation (IVF)
  • Coverage of cycles of intrauterine insemination
  • Coverage of fertility preservation cycles for all genders, including storage for five years
  • Coverage of surrogacy-related expenses
• 80% of local or international adoption costs, including legal fees, up to $20,000 CAD
• Top-up of government parental benefits: 12 weeks paid leave at 100% of salary, paid by the employer, for any member of a parenting team, including biological or adoptive parents, common-law partners and legal guardians.

Our goal is to place the employee at the centre of our decision-making process, start the discussion around parental benefit plans that include the LGBTQIA+ community and also inspire other companies to take that step too, so we create a more equitable society for parents.