Sikuli is a scripting language to automate the Web Application or desktop Application by using screenshots or the images of the Webpage or to control screenshots. It is Python based scripting to write visual automation scripts for automating Browser or desktop based Applications.
Sikuli was an open-source research project at the User Interface Design Group at MIT.
Sikuli is an automation tool, which is used to automate anything which you see on the screen or graphical user Interface (GUI). It's built using an Image Recognition algorithm to identify the controls on the Webpage or desktop Application. It is a bit similar to the Selenium automation tool, where we can identify a control in Selenum by its control properties like its Id, Tagname, Xpath, CSS Selector, Control name etc. But in such scenarios, if there are no control properties given for a control, we can use Sikuli to identify the control by its image.
You can learn how to automate Sikuli for desktop Application from Sikuli's official Website: http://www.sikuli.org/
Sikuli was an open-source research project at the User Interface Design Group at MIT.
Sikuli is an automation tool, which is used to automate anything which you see on the screen or graphical user Interface (GUI). It's built using an Image Recognition algorithm to identify the controls on the Webpage or desktop Application. It is a bit similar to the Selenium automation tool, where we can identify a control in Selenum by its control properties like its Id, Tagname, Xpath, CSS Selector, Control name etc. But in such scenarios, if there are no control properties given for a control, we can use Sikuli to identify the control by its image.
You can learn how to automate Sikuli for desktop Application from Sikuli's official Website: http://www.sikuli.org/
Sikuli can be developed using various technologies or languages like,
- .NET technologies(C# and .NET)
- Java technologies
- Javascript
- Ruby etc.
If we are developing Sikuli, using C# and .NET, Sikuli provides three different class libraries or APIs such as
- Sikuli Integrator (or) Sikuli Module
- Sikuli Sharp
- Sikuli4Net, which can be downloaded from nuget packages.
Among this three class Libraries; the most commonly used is Sikuli4Net, which provides various classes and methods to use it. It uses Java Runtime Environment (JRE) to execute the Sikuli scripts.
Follow the link to automate desktop Applications http://nadimsaker.blogspot.in/2015/02/sikuli-how-to-install-and-run-sikuli-in.html
Follow this link to automate Web Applications https://sourceforge.net/projects/sikuli4net/
Follow the link to automate desktop Applications http://nadimsaker.blogspot.in/2015/02/sikuli-how-to-install-and-run-sikuli-in.html
Follow this link to automate Web Applications https://sourceforge.net/projects/sikuli4net/
Following are the DLLl's required to run Sikuli, using Selenium in C#,
- Sikuli4Net DLL can be downloaded from NuGet packages.
- Newtonsoft.json can be downloaded from NuGet packages.
- Selenium webdriver can be downloaded from NuGet packages.
- Webdriver.Support can also be downloaded from NuGet packages.
- sikulirestapi-1.0.jar can be downloaded and placed into Unit testing class library solution.
Surendra ReddyPosted Feb 27, 2023, 12:20 PM
Code :APILauncher launch = new APILauncher(true); launch.Start(); System.Threading.Thread.Sleep(5000); Pattern Image1 = new Pattern(@"C:\SikuliImages\8.png"); Sikuli4Net.sikuli_REST.Screen scr = new Sikuli4Net.sikuli_REST.Screen(); scr.Wait(Image1, 10); scr.Click(Image1, true);
Surendra ReddyPosted Feb 27, 2023, 12:20 PM
I tried to using API but getting below Result: FAILorg.sikuli.script.Pattern cannot be cast to java.lang.String
Surendra ReddyPosted Feb 27, 2023, 12:19 PM
I am getting below error for creating session sikuli if you have idea pls let me knowException occured in SikuliSharpClient() Message : Java executable not found in expected folder: C:\***\****\bin\java.exe. If you have multiple Java installations, you may want to set the JAVA_HOME environment variable.
Surya GowthamPosted Mar 16, 2020, 2:13 AM
Hi i need to handle windows based pop up in my web application . example : to click allow button in the windows based popup . Kindly share your suggestions , if its possible to handle the above scenario using sikuli
Sunil SKPosted Dec 21, 2018, 3:27 AM
Hi Khaja, I am trying automate a window based application using Sikuli Selenium in C#. I followed all the steps you mentioned above but I am getting on click event. Also I have posted same question in fourms. PFA screen with below log. Error Screen: https://i.stack.imgur.com/65hQ1.jpg Please suggest solution. Log- :::Jar already downloaded, launching jetty server...::: :::Starting jetty server...::: :::Making Request to Service: http://localhost:8080/sikuli/api/click POST: {"jPattern":{"imagePath":"C:\Sikuli_Image\Start.PNG","offset_x":0,"offset_y":0,"similar":0.7},"jKeyModifier":"NONE"}::: :::{"result":"FAIL","stacktrace":"org.json.JSONException: JSONObject[\"imageIn64\"] not found.\r\n\tat org.json.JSONObject.get(JSONObject.java:473)\r\n\tat org.json.JSONObject.getString(JSONObject.java:654)\r\n\tat org.sikuliapi.rest.JPattern.getJPattern(JPattern.java:41)\r\n\tat org.sikuliapi.rest.JClick.getJClick(JClick.java:31)\r\n\tat org.sikuliapi.rest.servlets.click.doPost(click.java:34)\r\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:755)\r\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:848)\r\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:671)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:448)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:211)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:375)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)\r\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\r\n\tat org.eclipse.jetty.server.Server.handle(Server.java:449)\r\n\tat org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:246)\r\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:265)\r\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:589)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:520)\r\n\tat java.lang.Thread.run(Unknown Source)\r\n","message":"JSONObject[\"imageIn64\"] not found."}::: :::Result: FAIL Message: JSONObject["imageIn64"] not found. Stacktrace: org.json.JSONException: JSONObject["imageIn64"] not found. at org.json.JSONObject.get(JSONObject.java:473) at org.json.JSONObject.getString(JSONObject.java:654) at org.sikuliapi.rest.JPattern.getJPattern(JPattern.java:41) at org.sikuliapi.rest.JClick.getJClick(JClick.java:31) at org.sikuliapi.rest.servlets.click.doPost(click.java:34) at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:671) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:448) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:211) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1070) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:375) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1004) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:449) at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:246) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:265) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:589) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:520) at java.lang.Thread.run(Unknown Source) :::
Khaja MoizuddinPosted Sep 26, 2018, 4:30 AM
Yes Suchaganti, there is no kind of parallel testing with Sikuli I guess, it will search for the controls in the present focused browser.
Suchaganti CPosted Sep 21, 2018, 8:22 AM
Thanks for the post. Is there a way we could handle dual monitors? I think with the above code it would only search for an image in the primary screen.
dhananjay agrawalPosted May 14, 2018, 5:15 AM
SIMPLIV Offers to Learn More About Sikuli – Pattern-Matching and Automation Visit @ https://bit.ly/2wA2Hlv
Khaja MoizuddinPosted Aug 8, 2017, 11:39 PM
Post your question in forums, will help you out there
Khaja MoizuddinPosted Aug 7, 2017, 1:28 PM
Search using Nuget packages, you will find some related API'S to work in eclipse
viswaPosted Aug 7, 2017, 6:20 AM
Hi Khaja, Good and simple way of explanation. Do we have any method for exact find , I am able to find that with Eclipse version but not in C# wrapper.
Khaja MoizuddinPosted May 1, 2017, 5:47 AM
Please Safa V post your question in forums , i will help you out there.
V SafaPosted Apr 30, 2017, 11:34 PM
Thank you Khaja, Can you please help me about where should I put Sikuli Rest API? and I get : "Unable to connect to the remote server" exception. What can be the problem? Thank you