Robot Framework

Robot Framework is a framework that automates acceptance testing and acceptance test-driven development. Being generic in nature, the framework can also be used to automate business processes, often called Robotic Process Automation (RPA).

The core of Robot Framework is written in Python but libraries extending it can be in Python or Java. The framework is independent of operating system and application. It's open source and is sponsored by the Robot Framework Foundation.

Test cases in Robot Framework are written using keywords. Keywords themselves are abstracted away from their implementation. This promotes reuse of keywords across tests and easier maintenance of tests, particularly in large projects.

Discussion

  • What are the benefits of using Robot Framework?
    An example test case in Robot Framework. Source: Yurko 2018, fig. 1.
    An example test case in Robot Framework. Source: Yurko 2018, fig. 1.

    Since Robot Framework is keyword-driven, it has the benefit of separating high-level descriptions of tests from the low-level implementation details. Test writers may not be programmers. Since keywords are closer to English than programming, test writers find it easier to write and maintain tests. Keywords can be reused across tests.

    Tests are easy to read since they often take a tabular form. In this form, the first column contains keywords. Other columns contain arguments to each keyword. For example, "Create User" is a keyword that takes username and password as arguments. When implemented, this can translate to launching a web browser, clicking a button, entering the data, and confirming the request to create a new user.

    Keywords are composable. This means that keywords themselves can depend on other keywords. This allows us to create tests at different levels of abstraction.

    Since implementation is kept separate, test cases can be developed agnostic of any programming language. Testing can be planned at an early stage even before any implementation is ready.

  • What are the main features of Robot Framework?

    While keyword-driven tests are common, Robot Framework can also be used to create data-driven and behaviour-driven tests. It can also be used for automating any business process, and thus not limited to testing.

    Test cases can be organized into test suites. Execution can be based on test suites, test cases, or tags, all of which can be specified by name or pattern. Test suites and test cases can have setup and teardown.

    The framework produces neat reports and logs. It also allows for debugging.

    Developers can extend the framework with their own keywords and libraries.

    Test cases can be created in a single text file but they can also be created within the Robot Framework IDE (RIDE). Some popular IDEs can integrate with the framework including PyCharm, RED (Robot Editor) for Eclipse, and Robot Framework Intellisense for VS Code.

  • Could you list some built-in keywords used in Robot Framework?

    Online documentation gives complete descriptions of all built-in keywords along with accepted arguments. We list a few of them:

    • Conditionals: Keyword Should Exist, Length Should Be, Should Be Empty, Should Be Equal, Should Contain, Variable Should Exist
    • Control Flow: Continue For Loop, Exit For Loop, Repeat Keyword, Return From Keyword
    • Conversions: Convert to Binary, Convert to Boolean, Convert to Bytes, Convert to Integer
    • Executions: Call Method, Evaluate, No Operation, Run Keyword
    • Library: Get Library Instance, Import Library, Reload Library, Set Library Search Order
    • Logging: Log, Log Many, Log To Console, Log Variables, Set Log Level, Set Test Message
    • String Operations: Catenate, Should End With, Should Match, Should Match Regexp, Should Not Be Equal As Strings
    • Timing: Get Time, Run Keyword If Timeout Occurred, Sleep, Wait Until Keyword Succeeds
    • Variables: Get Variable Value, Import Variables, Set Global Variable, Set Suite Variable, Set Test Variable
    • Verdict: Fail, Fatal Error, Pass Execution, Pass Execution If
  • Where does Robot Framework fit within a test automation architecture?
    Architecture of Robot Framework. Source: QATestLab 2017, pg. 14.
    Architecture of Robot Framework. Source: QATestLab 2017, pg. 14.

    The job of the framework is to read and process data, execute test cases, and generate reports and logs. The core of the framework doesn't know anything about the system under test (SUT). Actual interaction with SUT is handled by various libraries. Libraries themselves rely on application interfaces or low-level test tools to interact with SUT.

    Let's take the example of testing a web application. Suppose Selenium is used for testing the application. Robot Framework doesn't directly control interactions with the web browser or databases. SeleniumLibrary and DatabaseLibrary are two libraries that manage these interactions. These libraries expose relevant keywords that can be used in test cases without worrying about how they are implemented.

  • Could you share some best practices when writing tests in Robot Framework?

    Test cases should be easy to understand. Use descriptive names for test suites and test cases. Likewise, keyword names should clear and descriptive. A common convention is to use title case for keywords. For example, use Input Text rather than Input text. For variables, use lowercase for local scope and uppercase for global scope.

    Document each test suite. Describe the purpose of tests in that suite and the execution environment. Include links to external documents. At test case level, documentation may not be required. The use of suitable tags is often more useful.

    Tests within a suite should be related. Don't have too many tests in a single file, unless they are data driven. Each test should be independent of others and should rely only on setup and teardown. Each test case should test something specific. A large test can possibly cover an end-to-end scenario.

    If there are user-defined keywords, document the arguments and return values. Between keywords, assign return values to variables, and then pass these variables as arguments.

    Avoid sleeps. Instead, wait for an event with a timeout.

  • What are some useful resources and tools for Robot Framework?

    The official Robot Framework website is a good starting point. Those new to writing tests, can study the examples there. The User Guide is another useful resource.

    Beginners should get familiar with some of the standard libraries including Builtin, OperatingSystem, String, Process, DateTime, Collections, Screenshot, and more. The framework has been extended by many third-party libraries. Choose what suits your application. There are libraries to interface or work with Android, Eclipse, databases, Selenium, Django, SSH, FTP, HTTP, MQTT, and more.

    Among the useful tools are Rebot for generating logs and reports; Tidy for cleaning or formatting test data files; Libdoc and Testdoc for documentation. Pabot is a tool for parallel execution. Robot Corder can record and playback browser test automation. You can choose among various editors and build tools. There's also a Jupyter kernel.

Milestones

2004

At the Helsinki University of Technology, Paul Laukkanen/Klärck starts looking into large-scale test automation frameworks. As part of his Master's thesis (published in 2006), he creates a keyword-driven prototype framework.

2005

Nokia Networks is on the lookout for a generic test automation framework. Robot Framework is born based on Klärck's work.

Jun
2008

Nokia Networks open sources Robot Framework with the release of version 2.0.

Dec
2012
RIDE screenshot showing the use keywords and their arguments in a test case. Source: Goralight 2016.
RIDE screenshot showing the use keywords and their arguments in a test case. Source: Goralight 2016.

Version 1.0 of Robot Framework IDE (RIDE) is released on GitHub. RIDE simplifies the writing and execution of automated tests. Earlier version v0.40, also available on GitHub, can be traced to January 2012. Code before this was hosted at Google Code.

Dec
2015

Robot Framework 3.0 is released. With this release, Python 3 is supported.

Nov
2016

Standard ISO/IEC/IEEE 29119-5:2016, Part 5: Keyword-driven testing is published. This underscores the growing importance of keyword-driven testing.

Jan
2018

Called RoboCon, the first annual Robot Framework Conference is organized in Helsinki, Finland.

Dec
2018

Robot Framework 3.1 is released. With this release, Robotic Process Automation (RPA) is supported. This means that the framework can be invoked to automate tasks and not just tests.

Sample Code

  • # Source: Adapted from Example 5 at https://robotframework.org/#examples
    # Accessed: 2019-07-31
     
    # Install CalculatorLibrary from https://github.com/robotframework/RobotDemo
    # Tested with CalculatorLibrary version c7d226f of Jan 25, 2019
     
    *** Settings ***
    Test Template     Calculate
    Library           CalculatorLibrary.py
     
    *** Test Cases ***    Expression    Expected
    Additions             12 + 2 + 2    16
                          2 + -3        -1
     
    Substractions         12 - 2 - 2    8
                          2 - -3        5
     
    Multiplication        12 * 2 * 2    48
                          2 * -3        -6
     
    Division              12 / 2 / 2    3
                          2 / -3        -1
     
    Calculation error     [Template]    Calculation should fail
                          kekkonen      Invalid button 'k'.
                          ${EMPTY}      Invalid expression.
                          1 / 0         Division by zero.
     
    *** Keywords ***
    Calculate
        [Arguments]    ${expression}    ${expected}
        Push buttons    C${expression}=
        Result should be    ${expected}
     
    Calculation should fail
        [Arguments]    ${expression}    ${expected}
        ${error} =    Should cause error    C${expression}=
        Should be equal    ${expected}    ${error}
     

References

  1. Aro, Ismo. 2018. "Robot Framework Foundation: Welcome to RoboCon." RoboCon 2018, via YouTube, February 15. Accessed 2019-07-30.
  2. Bristiel, Laurent. 2014. "What editor for Robot Framework test cases." August 17. Accessed 2019-07-30.
  3. Eficode. 2016. "Robot Framework: Past, Present and Future." Blog, Eficode, June 20. Accessed 2019-07-30.
  4. Goralight. 2016. "How to handle windows authentication popup in Robot Framework RIDE?" StackOverflow, December 08. Accessed 2019-07-30.
  5. Jaspers, Thomas. 2012a. "Robot Framework IDE (RIDE) – Overview." Blog, codecentric, January 13. Accessed 2019-07-30.
  6. Jaspers, Thomas. 2012b. "Robot Framework Tutorial – A complete example." Blog, codecentric, April 27. Accessed 2019-07-30.
  7. Klärck, Pekka. 2009. "Experience." Eliga Oy. Accessed 2019-07-30.
  8. Laukkanen, Pekka. 2006. "Data-Driven and Keyword-Driven Test Automation Frameworks." Master’s thesis, Helsinki University of Technology, February 24. Accessed 2019-07-30.
  9. Nguyen, Thuc. 2019. "Keyword-Driven Testing: The Best Practices You Can’t Afford to Miss." Blog, LogiGear Corporation, February 25. Accessed 2019-07-30.
  10. QATestLab. 2017. "Guide to Test Automation Tools 2017-2018." Whitepaper, QATestLab. Accessed 2019-07-30.
  11. Robot Framework Foundation. 2019. "Robot Framework User Guide." Version 3.1.2, May 24. Accessed 2019-07-30.
  12. RobotFramework. 2019. "Homepage." Accessed 2019-07-30.
  13. RobotFramework Docs. 2019. "BuiltIn." May 24. Accessed 2019-07-30.
  14. RobotFramework GitHub. 2015. "Robot Framework 3.0." Release Notes, December 31. Accessed 2019-07-30.
  15. RobotFramework GitHub. 2019a. "robotframework/robotframework." May 24. Accessed 2019-07-30.
  16. RobotFramework GitHub. 2019b. "robotframework/RIDE." July 22. Accessed 2019-07-30.
  17. RobotFramework GitHub. 2019c. "Robot Framework 3.1." Release Notes, April 16. Accessed 2019-07-30.
  18. RobotFramework GitHub. 2019d. "How to write good test cases using Robot Framework." HowToWriteGoodTestCases, April 05. Accessed 2019-07-30.
  19. Soukka, Asko. 2018. "RoboCon 2018 and Robot Framework Jupyter support." January 27. Accessed 2019-07-30.
  20. Wikipedia. 2019. "ISO/IEC 29119." Wikipedia, April 26. Accessed 2019-07-30.
  21. Yurko, Dmitriy. 2018. "Keyword-Driven Testing with Robot Framework." Blog, Apriorit, September 20. Accessed 2019-07-30.

Further Reading

  1. Jaspers, Thomas. 2016. "Robot Framework Tutorial 2016." Blog, codecentric, January 08. Accessed 2018-07-31.
  2. Ihnatsyeu, Dzmitry. 2018. "Robot FrameWork: The Ultimate Guide to Running Your Tests." Blog, BlazeMeter, July 18. Accessed 2018-07-31.
  3. Robot Framework Foundation. 2019. "Robot Framework User Guide." Version 3.1.2, May 24. Accessed 2019-07-30.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
3
0
1673
1276
Words
4
Likes
11K
Hits

Cite As

Devopedia. 2019. "Robot Framework." Version 3, August 1. Accessed 2024-06-25. https://devopedia.org/robot-framework
Contributed by
1 author


Last updated on
2019-08-01 05:30:06