Flow Builder: Count LEGO Formulas by Color

Overview

In this tutorial you will create your first working application that can process an image and detect and count objects in it, showing the results on a dashboard.

What you'll learn

• How to use Flow Builder to create a business logic
• How to use an AI Model node
• How tu use a Virtual Camera
• How to use a Counter node
• How to use an Accumulator node
• How to create a dashboard to show the results
• How to start and stop a deployment
• How to show a dashboard

What you'll build

By the end of this tutorial you will have a working application that can detect objects in an image, count them and show the results on a dashboard.

Prerequisites

To complete this tutorial you wil need the following:

Required assets:

• The example image archive emitter_od_formula1_1920x1080_dataset.zip that you can download from here
• The password for the platformadmin user, that you received via email when you installed the platform.

To complete this tutorial, you need the platformadmin role. If you don't have this role, then you cannot create or modify business logic.

Estimated time

60 minutes

Step 1 : Create an Image Dataset

In the Hub, create a new image dataset that will be used later by the virtual camera.

1. Click on Datasets in the side bar. The datasets page opens.

2. Click on the Create Dataset button. The Create Dataset dialog appears.
3. Type My first dataset In the Name field
4. Optionally add a description in the Description field

5. Click on the Confirm button. The empty dataset My first dataset is shown.

6. Click on the Upload Files button. The Upload files dialog appears.

7. Drag and drop the emitter_od_formula1_1920x1080_dataset.zip file in the dialog. The file is ready for uploding in the platform.

8. Click the Upload button to begin the upload.
9. A progress bar show the uploading status

10. At the end of the process, the My first dataset page is shown, populated with the images in the zip file.

Step 2 : Create a Dashboard

In this step you will create the dashboard to visualize the output of your business logic.

1. Click on Builder in the sidebar. This will open the Builder interface.

2. Click on Create project to create your first project. The Create project dialog appears. Type My first project in the Name field and optionally add a description in the Description field.

3. Click Create. The project is created.

4. In the sidebar, click on Dashboard Builder. The builder appears.

5. Click on the + button. In the Create dashboard dialog type My first dashboard in the Name field and optionally a description in the Description field.

6. Click on Create to create the dashboard.
7. From the widget list, drag and drop the Stream widget into the canvas. The widget is positioned on the canvas.

💡 You can adjust the widget's size by dragging the bottom right corner.

8. Click on the kebab menu (the three little dots in the top right corner of the widget), then click on Edit to configure the widget.

9. In the Stream Widget Configuration:

   • Set the Title to Camera View
   • Activate the Expandable checkbox

10. Click Save to store the changes.
11. Drag and drop a Value widget on the canvas.
12. Click the kebab menu. Click Edit to open the configuration panel. Set the Title to Frame Count and click Save.

13. Add 4 new value widgets to the canvas. Name them:
    • Blue
    • Red
    • Blue Total
    • Red Total

Repeat steps 11 and 12 per each new widget.

14. Click Save to save your new dashboard.

💡 Create your dashboards first, and the business logic next. This will facilitate your development.

Step 3 : Create your first flow

In this step, you will create your first flow, starting with the input and the output nodes.

1. Click on Flow Builder in the sidebar.
2. Click on the pen icon at the top of the page on the left.
3. In the Rename flow template dialog type My first flow in the Flow name field and click Save to save the changes.

4. Click the Save button at the top-right of the page.

💡 Save the changes to the flow often. If you leave the page without saving, all unsaved changes will be lost.

5. From the node list on the left, select the Image Stream node in the INPUTS group and drag it on the canvas. The node will appear on the canvas.
6. Type VideoIn on the Custom name field of the node to update its name.
7. From the OUTPUTS group, add the Image node to the flow. Name it VideoOut.
8. From the OUTPUTS group, add 5 Number nodes to the flow. Name them:
   • FrameCountOut
   • BlueOut
   • RedOut
   • BlueTotOut
   • RedTotOut
9. Left-click on the image ouput of the VideoIn node and, without releasing the mouse button, connect it to the image input of the VideoOut node.
10. Click Save to save the changes to the flow.

You have just created your first, very basic, flow template. It sends the image received from the camera by the VideoIn node to the VideoOut node, that will provide it to the dashboard.

The remaining unconnected output nodes will be used later in this tutorial.

Step 4 : Create your first device

In this step, you will define the camera device that provides the video input to the VideoIn node.

1. From the sidebar, click on Devices. The device page appears.
2. Click on Add device to create your first device.
3. In the Add new device dialog:
   • Select virtual camera from the Device type drop-down menu
   • Type Cam01 in the Device name field
   • Leave the default value 1 in the Camera FPS field
   • Activate the Run virtual image feeder in loop checkbox.

4. Click Add to add the new device to the list of project's devices.

Step 5 : Create your first deployment

You are now ready to create your first deployment, connecting all the pieces together to create a working application.

1. Click Deployments on the sidebar
2. Click Create Deployment to create your first deployment
3. In the Create deployment dialog:
   • Type First Deployment in the Name field
   • Optionally add a Description
   • Click Create to create the deployment

4. Click on the + icon to add a flow template to the deployment
5. In the Select Flow dialog, click on My first flow to select it.
6. Click on Add Flow to add the flow to the deployment.

7. In the menu on the left, select Devices
8. Associate the VideoIn input node with the Cam01 device. Check the Immediate check-box.

9. Click Next. The configuration switch to the configuration of the Dashboard's connection.
10. Associate the Camera View widget with the VideoOut node.

11. Click on Flow Config, then click on Done

Step 6 - Start the deployment and visualize results

You are now ready to start the deployment and check the output.

1. Click on Start to start the deployment.
2. in the CAUTION message, click on Start deployment to confirm your intention to start the deployment.

3. The deployment is started.

4. Now switch back to the Hub interface. Go to the Overview page to see the current status of your running deployment.

5. Click on Virtual Image Feeder on the sidebar.
6. Associate the Cam01 device with the dataset My first dataset

7. Click on the Start icon to start the camera. The camera will start streaming.

8. Click on Dashboards on the sidebar.

9. Click on My first dashboard on the list of available dashboards.
10. Your dashboard appears and you will see a video stream in the Camera View widget.

Step 7 - Add a ML Model to the flow

The next step will be to add a model to the flow. Your goal is to detect blue and red cars in the images, and draw a blue and red box around them.

1. Switch to the Flow Builder
2. Click on Deployments
3. Click on Stop to stop the running deployment
4. In the CAUTION message, click on Stop deployment to confirm your choice.

5. Click on Models
6. Click on Add model to project
7. Click on From remote store
8. In the Remote models list dialog:
   • Select km-public-model in the Customer list
   • Select OD_1920x1080_Formula1 (version: 1) in the Select Model

9. Click on Download
10. The model is downloaded in the local model store and is now ready to be used in a flow.

11. Click on Flow Builder
12. From the MODELS group, add the AI Model to the flow. Name it Detector
13. Click on the cog icon on the top right corner of the flow. In the node configuration panel:
    • Select km-public-model/OD_1920x1080_Formula1:1 in the AI Model Name
    • Select unpacked in the Output format

14. Click on the X to close the node configuration panel.
15. From the MODEL PROCESSORS group, add two Annotate polygons node to the flow. Name them:
    • BluePol
    • RedPol
16. Click on the cog icon of BluePol
17. Use the color picker to configure Polygon Color to blue.

18. Configure the Polygon Color of RedPol to red.
19. Connect the blue_polygons output of the Detector node to the polygon_list input of the BluePol node.
20. Connect the red_polygons output of the Detector node to the polygon_list input of the RedPol node.
21. From MODEL PROCESSOR group, add a Merge Polygons node to the flow. Name it BlueRedPol.
22. Connect the output of BluePol to the input input_polygons_0 of BlueRedPol
23. Connect the output of RedPol to the input input_polygons_1 of BlueRedPol
24. From MODEL PROCESSOR group, add a Draw Polygons to the flow. Name it AllPol.
25. Connect the output of BlueRedPol to the polygon_list input of AllPol
26. Click on the connection between VideoIn and VideoOut. Click on the X to remove the connection.

26. Connect the output of VideoIn to the input_image of AllPol
27. From IMAGE PROCESSORS group, add a Resize Image node to the flow. Name it Resizer. In its configuration panel:
    • Set Width to 1920
    • Set Height to 1080

28. Conect the output of Resizer to the input of Detector
29. Connect the output of VideoIn to the input of Resizer
30. Connect the output of AllPol to the input of VideoOut
31. Click Save to store all these changes.

Your flow should now be like the one in the image below.

32. Click on Deployments and start the First Deployment
33. Go to the Hub and, in the Virtual Image Feeder, restart the Cam01 video feed.
34. Go on My first dashboard. You will see blue and red polygons around the cars, depending on their color.
35. Click on the Camera View widget to zoom it.

Step 8 : Count Frames and Polygons

In this step, you will learn how to use a counter node.

1. In the Builder, stop the running deployment.
2. Click on Flow Builder
3. From the AGGREGATORS group, add a Cunter node to the flow. Name it FrameCount
4. Connect the output of VideoIn to the input of FrameCount
5. Connect the output of FrameCount to the input of FrameCountOut
6. From the MODEL PROCESSORS group, add two Count Polygons nodes to the flow. Name them:
   • BlueCount
   • RedCount
7. Connect the output of BluePol to the input of BlueCount
8. Connect the output of BlueCount to the input of BlueOut
9. Connect the output of RedPol to the input of RedCount
10. Connect the output of RedCount to the input of RedOut
11. Click Save to store the changes.

10. Open the Deployment editor
11. Click on Configure to modify the configuration of the First Deployment
12. In the Dashboard page:
    • Associate the FrameCount widget with the FrameCountOut node
    • Associate the Blue widget with the BlueOut node
    • Associate the Red widget with the RedOut node

13. Restart the First Deployment
14. In the Hub, restart the Cam01 virtual video feed
15. Reopen the My first dashboard.

You will now see counters running according to the logic we implemented. The total frame count is displayed in the Frame Count widget. The Blue and Red frame show the number of blue and red cars detected in the current frame.

Step 9 : Accumulate the values

In this step, you will complete the logic by adding the blue and red cars detected in each frame to a counter of total red and blue cars detected since the deployment's start.

1. In the Builder stop the running deployment
2. Switch to the Flow Builder
3. From the AGGREGATORS group, add two Accumulator nodes to the flow. Name them:
   • BlueTot
   • RedTot
4. Connect the BlueCount output to the BlueTot input input
5. Connect the BlueTot output to the input of BlueTotOut
6. Connect the output of RedCount to the input input of RedTot
7. Connect the output of RedTot to the input of RedTotOut
8. Click Save to save the updates

9. In the deployment editor, open the configuration of First Deployment
10. In the Dashboard section, associate
    • Blue Total widget with BlueTotOut node
    • Red Total widget with RedTotOut node

11. Restart the First Deployment
12. Restart the Cam01 virtual image feeder
13. Refresh My first dashboard

You will now see the total number of red and blue cars accumulate in the Blue Total and Red Total widget.

Verify your work

You've completed the tutorial successfully if:

• Images are streaming in the Camera View widget
• Blue cars in the images are annotated with a blue polygon
• Red cars in the image are annotated with a red polygon
• The counter in the Frame Count widget increments by 1 at each image
• The counter in the Blue widget is equal to the number of blue cars in the image
• The counter in the Red widget is equal to the number of red cars in the image
• The counter in the Blue Total widget is incremented by the number in Blue widget at each frame processed
• The counter in the Red Total widget is incremented by the number in the Red widget at each frame processed.

---

Summary

In this tutorial, you learned how to:

• Create an image dataset
• Create a project
• Add devices and machine learning models to a project
• Create a flow template and use nodes to create a business logic
• Create a dashboard to visualize the results of the business logic
• Create, start and stop a deployment
• Associate devices with input nodes
• Associate output nodes with dashboard widgets
• Create a virtual camera device and associate it to an image dataset
• Start and stop a virtual camera

You also learned how to switch between the Hub, the Builder and the Dashboards and how to move in them.

Next steps

Now that you've completed this tutorial, you can:

• Explore more the Flow Builder
• Learn more about the available nodes
• Learn more about Video Vision

Related documentation