Note: Do not capture an area that is too big as it will make the computer vision process more complicated and will take longer to complete. We will select our Submit button by drawing a rectangle around it.

An experience will launch that will allow you to capture the area of the screen that you want to be able to detect.

Note: Be sure to have the web page, or application, that you are automating launched before trying to capture the image. The first step that we need to perform is to click on the Capture new image button. We now need to configure our Wait for image action. We can find the Wait for Image action in the left navigation panel by searching for it.

If we want to click this button, we can use the Wait for Image action to detect the image and then we can use its co-ordinates to subsequently send a mouse click.

This could be because the web page developer is using dynamic values to generate the button or perhaps it is nested in iframes, divs or other markup that makes locating it more difficult.

Inside of Power Automate Desktop (PAD) we can use a computer vision strategy by using the Wait for Image action.Ĭonsider a situation where we may have the following form and we are having trouble locating our Submit button using xpath, css or other selectors. When encountering situations where you can’t locate a specific object, you may defer to other strategies including hot keys (CTRL + S), sending keys like Tab/Enter or using computer vision.Ĭomputer Vision will look for specific objects by examining a screen and comparing pixels to an image that you capture and store inside of a repository in Power Automate Desktop. This includes locating specific objects during recordings.

When building Robotic Process Automation (RPA) processes, it may be difficult to locate certain objects, depending upon the underlying technology and vintage of the application.

Power automate web automation