Continue Thinking Small: Next level machine learning with TinyML — Maria Jose Molina Contreras

Thank you very much.

It's, I'm super happy to be here with all of you talking about tiny mail.

Before I start, I would like to just let you know which kind of talk will be this one.

Just to be sure that everyone is in the same page.

This is an introductory talk where we are going to cover a lot of topics.

I'm going to try not to overwhelm you a lot, but we're going to see a lot of things.

And you're going to discover why the structure of this talk, during the talk.

Well, I'm going to keep it very short because the third bit of great job introducing me.

Then I'm just going to go into a couple of things.

In the last years, I have been working mainly with center data.

And it's this kind of projects that I just love to build also in my free time.

For instance, growing lettuces in my balcony.

It's something that I just enjoy a lot and I truly recommend to you do it.

Awesome.

Implementing some machine learning, pipelines to increase your happiness plant.

And well, as you can see, my plants needed.

And it's a clear example that sometimes we don't need machine learning to understand that our plants are not happy.

But I enjoy a lot doing these kind of things.

And in my last project that I developed at home, it was for kind of half control of my air quality.

And I have a lot of fun building that because it was a monitoring system, but also a predictive system.

I could know when I should activate my ventilation system that in other words was just when opened a window.

But I and I enjoy a lot.

I thought, wait.

What if I can have this kind of system?

But with me, when I go to the street and see who is the contamination level,

what if I need to wear a mask or not?

What do these kind of things?

And that was the moment when I start with tiny ML.

Currently, I am developing this project.

I cannot show you yet because it's not done.

But maybe for the next conference, we'll be ready.

But during this time developing this project, I learned a lot of things.

And I realized that this knowledge could be shared with a community.

And maybe it could help to someone to just do it easier.

And let's start for the beginning.

What is tiny ML?

Tiny ML.

It's an interception of two great goals.

It's an interception between data science and electronics.

And specifically between machine learning and embedded system.

And wow, it's awesome because you have a very great two worlds working together.

Yeah, that's true.

However, it could be kind of challenging sometimes because you need some knowledge of data science

and also some knowledge of electronics.

Or have two teams that can work together and have good communication between them.

But we are going to cover all these topics in case that you can just build your own projects as well.

Why, Tiny ML?

Well, one of the key points from my perspective is data privacy because you're going to be able to have all the

manage your data, your prediction in the device without having the necessity to have access to a cloud or other things.

Low energy consumption.

This device doesn't require a lot of energy and a lot of power.

Low latency is how it should be faster.

And also connectivity.

If you're working with IoT systems and data, this kind of sensor data,

maybe you are going to face some internet connections time to time and you're going to lose data,

or you're going to have issues, or maybe you need to have this system in a place where the internet doesn't arrive.

This is going of the ones.

And maybe you're thinking, yeah, but has this application in real life?

Yes, there are applications.

We can find examples currently and also for the future.

One of the topics that also I think that it's very, very important, it's healthcare.

And it's because the data privacy that we were talking about.

If you have a device that track your phone information and can keep this data privacy,

maybe it's a good way to make things.

Of course, this is only an example, a publication, but there aren't many of them that we can discuss later on.

Also, I get a culture.

Maybe you're thinking, why I get a culture scheme?

Well, not everywhere arrive.

Internet, not everyone have access to internet there and has the ability to have a device or even in the mobile.

Some models for predicting pairs, predicting who happened to your plant go help to the farmers around the world.

Also, as matter of spaces, it's important to understand if you have a person in the room or not for security reasons,

sometimes in some spaces.

Then we have more.

We have predictive maintenance.

This is one of my favorite topics that I'm going to do in some mentioning because I have been working for long.

And it's because of that is that you can predict also if something going to fail in places that maybe the internet connection doesn't arrive.

Also, we have world conservation, there are projects that currently we can track elephants in this case, but also there are other with whales.

In this case, it's to avoid fortives, and avoid that the elephants could be killed.

And also, we have some recognition.

Maybe it's familiar to you this, hey, blah, blah, blah, blah, blah, blah, blah, blah.

And something happened.

Maybe something happened.

Maybe something happened.

Maybe some stone could play.

Yeah, maybe you're familiar with some of those.

This is also an application or something.

Then as we were talking about before, there are some challenges or things that you need to have in consideration before start this kind of projects.

And we are going to start from the embedded system first, from the electronics part.

First is that the development boards, which one you are going to use, whatever the needs of your project, which more.

Of course, this is my bias because it's the boards that I am using currently for developing my undoing my experiments.

And specifically, I am playing around these two, but of course you can choose the board that you need or you want.

What is important?

Understand what are your needs of your project.

And then this means that you're going to need to always have in mind what is your data science question on problem.

And we are going to see it later in the machine learning part because even it's very cool to see I'm going to take this board because it's trending topic.

Maybe it's not the best for your business case or your experiments or whatever you want to implement, right?

We are going to see later on.

But just let you know, I'm super fine of those because also it's super small.

I'm not sure if you can see it, but because it's very small, it's the size of my A.

But just to give you my money to sense of that.

What are some of the challenges that the region is seeing?

It's very hard.

It's very hard to have some standards and tell you you need to do ABC for all the boards.

And of course, if you are in the electronic goal, for you will be very easy.

But if you came from the data, it will be more challenging.

And of course, the constraints, the constraints that we are going to find.

And this is something very important because when we're developing things in data science, of course,

we try to be efficient.

We try to optimize.

However, we have other kind of challenges in the computing side, for instance.

Because usually, we develop our project in a computer or in a cloud and kind of happy.

We train the models.

We have these things.

But then in this case, we'll be much challenging.

This means that, for instance, a training, we are going to need to keep it as we are doing it right now in a computer or in the cloud.

Of course, these numbers are audientities.

They're going to be changing the pens of the case.

Also, this is the information that came from this website.

Of course, it could be different.

But just, I wanted to share with you the constraints that we need to face.

And one of the key things is the memory.

We are going to need to fit machine learning models into this, into this storage.

And have it running in this memory.

But no worries, we are going to go into that.

But it's a real challenge.

Of course, development environment.

If you develop a new work with microcontroller for you, this is not a problem at all.

But if you came from a different world, it's like, okay, oh, oh, I do that, right?

This is new.

It's like, I need to connect it and imagine it happened, or I can do something else.

Yeah.

Well, there are different platforms that are different options that you can develop.

Of course, you can also use the command line.

Very simplified.

Or just just a platform that it's open source and also is language agnostic.

And you can work a lot.

And of course, you're going to know a bit of electronics or ask for some super with

information.

Because you're going to need to do some, the flashing and move files.

And understand how fun works and the specific to see of the board and these kind of things

that, again, when you are in the field is super obvious.

But when you came out of that, it's like, I'm not sure what I'm doing.

And of course, always within the documentation.

So check.

My recommendation is to be sure that you're going to be able to follow up.

But it's true that once you do a start, you realize that this, it's going to be a straightforward.

It's just a follow up, some steps and everything going to work sooner or later.

So for the machine learning challenges.

And here we're going to find some of them.

Well, I'm here.

We have a typical data science machine learning pipeline with the different steps.

If you are into the data science world, you know that this linearity is not correct.

We are going to need to collect the data, process the data, wait.

The model is not performing very well.

You're going to need to.

And then what about now?

I know wait any more data.

The data is not very good.

And you're going to need to have this iteration in continuous.

And this is going to happen the same.

This will be exactly the same.

You need to do exactly the same, the things that you do normally.

But with some differences that we are going to write later on.

The first, the, the, the, the, the only thing that I wanted to add in that point is like,

just be careful because as I was mentioning called the pipeline,

gonna need to be into the computer or cloud that you use until the deployment,

of course, that will be in our microcontroller.

If some of you are thinking, but wait, how I'm going to implement now neural network,

because I came from a different field.

How can I do that from zero?

There are options that will simplify this process.

And you can use a non code option.

In this case, in this platform, the, it's impulse, you can have all the pipeline.

Just with clicking options.

You can have the dashboard.

There is, you can have the collection of data that you can,

I think that you can have it with sound.

Then you can have some model, train the model, have the metrics and that.

One of the things that are very cool is that also you can know

how big you're gonna be your model when you train and see all your metrics.

On the side, if this is a good option for you, or you need to just now go back

and do all this pipeline that we were talking.

Maybe you need more data, maybe, maybe not.

One of the things that this platformer, I found it that was not matching

with my necessities, is that it was a specific, for some specific boards.

And I wanted to work with some specific boards, or Michael, not developing.

Obviously, in case that the availability of the models, because I want to solve

my data science problem in our specific way.

And in this case, the offer is still in the belloping, I think.

I think at some point they're gonna have it, but it's still that.

But if you want to try it, it's a very nice option for sure.

However, today, we are gonna go for trying to do it by scratch,

because I wanted to share all the points that you're gonna need to have in mind

once you have, once you want to develop the project to make decisions.

First thing, gonna be what type of data science problem I have.

It's a super-wise, or is it a super-wise?

Will be some classification, some progression.

What exactly will be an anomaly detection?

All these kind of questions you are gonna also need to answer.

And of course, depends on the type of data, you're gonna have different challenges.

For instance, for anomaly detection, in this case, they were, in this case,

were used in audio, and in other cases, vibration.

This means that the way that you're gonna need to process this data is completely different.

And work without you, for instance, is very challenging.

And you're gonna need to face some challenges that you are not gonna need to face it in vibration data, for instance.

Or if you are working with object recognition, with images, also will be different.

Of course, maybe you're thinking, okay, but then let's call like data from zero.

Yes, this is an option.

Then you have more options inside this option.

You can have sensors.

In this case, I'm sharing with you the sensor of CO2 from the project that I was mentioning before.

But also, you can have some microcontrollers with sensor integrated in the one that I was sharing before.

But really you're thinking about the way they have a mobile phone.

I can call like data from there.

Yes, of course you can, but also you're gonna need to handle them.

The storage or who are you gonna connect these or other challenges.

But of course, there is a full full of options.

What more?

We have public data sets.

We can also use them.

We can also use pre-training models.

People is the training models.

Google people, for instance, has models available that we can use and use transfer learning and do some kind of adjustment of our case.

Yeah, this is a possibility.

However, you're gonna need to also understand which kind of data and if this is something that works for you.

But yeah, there are a lot of options that we can have.

Let's go for the designing model and training the mobile.

Because, of course, you need again to make decision.

Don't be overwhelming for this slide because it's full of information,

but I want just to show you a couple of things that are super, super important to have in consideration before the start.

Let's imagine, for instance, that you have a case, a business case that you say, yeah, this, the solution.

It's a neural network because I need to implement a neural network.

Okay, then you can go to the supported models and then you need to decide which gonna be the library for the training in your computer or in the cloud.

And then just say, okay, let's go for tensile flow.

But then you're gonna also need to check if your microcontroller that we were talking at the beginning is compatible with this libraries.

And in the framework that we are gonna need to use in this case for tensile flow,

gonna be tensile flow like micro or, for instance, also is compatible with utensil depends of the platform that you're using.

Then this is quite a more complex that we were discussing at the beginning.

Yeah, choose a microcontroller that works for you.

Yes, works for you, but you need to have very good the final data science project and how you are gonna solve the problem.

And as we were mentioning before, as we have also constraints of memory and storage, keep it simple.

As simple you can solve your problem better.

But sometimes, you know, you cannot simplify more and there are problems that needs to be.

So of course more complexity, but this is my recommendation always.

But imagine, for instance, that you are using scikit learned because it's a library that it's awesome for data science.

And you want to continue doing your project with scikit learn.

Can I do that? The inference in a microcontroller. Yes, you can.

You can use the scikit learn for different sense.

Or as we were mentioning for tensile flow, there is a lot of options.

Use the mention that in case of tensile flow, we have tensile flow light.

And then we have tensile flow micro in the case of tensile flow light is more for mobile.

And for microcontrollers, it's the micro.

Just to give you this a small detail.

And then imagine, you train your neural network because you were, imagine that you were analyzing this case of detecting.

If you're in the room, there is a person of the, there is a cat.

Imagine that you are in a building, you are the security member and you need to know that time to time the alarm sound.

And you need to know if it's just a cat or is a person who is in the building.

And you implement the system to just have that.

And then you implement your neural network.

Everything is going very small, very good in your computer.

And you know, sometimes things only work in our computer and this is a problem.

And then we have another point.

And we have the demonstration because we need to feed all this neural network with all these layers, all these parameters, everything that was super good, our accuracy, all our metrics were very good.

In this microcontroller, that is super small and has constraints that we were mentioning before.

One of the, the most popular option is to use quantification, even I also added the weight planning.

In the physical quantification, what we are doing is reduce the, the precision of the weight from 32 bits to 8 bits.

And maybe someone has concern right now, or maybe not, but you should have some concerns.

Because if we have this size, this information, imagine that this is information from one side to another side of my hand.

And we reduce it to this with this quantification, we are losing some information.

And maybe it's not the most important, but maybe some of this information that we are taking out.

It's important for our business, for our model.

Then what is needed is to check before the deployment.

We need to check if the performance of our model when we were training it in our computer with everything, happy, everyone, very happy with the success of this model is the same after the quantification.

And also we are going to need to think if we are eager to sacrifice a bit of accuracy in terms to have this model smaller.

Because maybe if we are going to do this for healthcare, we need to be very careful.

Because if we have, I don't know, some accuracy that you accept as a good.

Also, once you have this quantification process, you also need to have the same feeling that this is good.

Well, the next challenge is that the deployment of the model person.

And of course, the inference.

But the deployment, the physical deployment that we were mentioning before, should not be a problem.

Once you know how to move one file from one to our microcontroller.

Then where is the challenge?

In the case of how temporal light, let's think in this case, they recommend that this is from the official documentation.

And the recommended is compared to our model to see array.

And then run the inference in C++.

And then it's like, okay, well, what now?

Because maybe for all of you, right C++ code is something that you are doing every day.

But for other people, maybe it's not.

And I have a new challenge, right?

Okay, now I need to learn C++ to write the code.

What is very nice is the skills to learn a new programming language.

This was a very nice motivation for me.

However, for some other people, it could be something that is a challenge.

And I was wondering, is possible to run the full pipeline with Python?

Well, that is the option to extend the macro Python in C.

This is from the official documentation.

And then what I did is try to find some projects of people who did that before.

And I found this person who provides some examples, the running TensorFlow with micro Python.

There are only three examples, but it's something that apparently is working for some specific boards also.

Then this is the workflow options that we have currently.

We have our training and optimization step with Python.

But then the inference needs to be low-level programming languages.

Or go to the inference with micro Python.

That we are still on that.

I think because it's the only case that I found some information.

Then the question will be why are there more Python options for inference?

And this could be a debate.

Perfectly, do you have any idea or comment someone who encoded to say,

why we don't have more pipeline, the inference is in Python?

Okay. Any other idea?

Yeah. I completely are with Path of view.

I think that it's a think of optimization and could be other things like libraries.

And I think that could be more things.

Sorry. Sorry, sorry.

The thing is that this is the perfect opportunity for two things as I see it.

First, we see that you can implement a workflow and project that can work very good between Python and other programming languages.

And communities can interact and we can learn from each other.

And on the other side, we can kind of open the door to see if we can find collaboration between the Python community members.

And see if we can kind of contribute to see more Python in this step.

And of course, always trying to be open-minded and accept the limitations that we could have.

With that reflection, I would like to finish my presentation.

Of course, my information in case that you want to discuss about this reflection,

or you need something of this whole pipeline.

I will be very happy to share knowledge or discuss with you.

Thanks a lot.

Thank you very much.

Thank you.

Thank you.