This content originally appeared on Telerik Blogs and was authored by Peter Vogel
If executing scheduled tests on your Test Studio computer (or just executing long-running tests on your computer) is getting in your way, you can set up an execution server and offload test runs to it.
Eventually, you arrive at a point where, working with Test Studio, you’ve created lots of end-to-end tests and you’ve been scheduling them to run on your Test Studio computer when you’re not around. That’s great … but it’s also created a new set of considerations to work through.
For example, tests are failing because you didn’t leave your computer in the “right” configuration for your tests to run (which means, of course, you have to re-run last night’s tests on your computer the next morning). In fact, to support some tests, you’ve had to configure your computer in ways that make no sense for your day-to-day work (which leads to another symptom: you have to reset your Test Studio computer after a test run before you can start work).
Plus, there’s the odd time that you have to run a test right now, rather than wait until the scheduled run, and that ties up Test Studio (especially annoying with long-running tests).
Furthermore, as you continue to add new tests, your tests are finishing closer and closer to when you come in to work. Even with headless browser testing, it’s only a matter of time until there aren’t enough hours in the night to run your tests … that is, if you keep running those tests on your Test Studio computer.
In other words, you’ve got to the point where you need an “execution server”—some computer where you can offload tests (scheduled or otherwise) from your Test Studio computer. And that’s what remote execution/scheduling is all about: Getting your Test Studio computer back for your own use. That this will also let you simultaneously test in multiple environments and run tests in parallel to get done faster is almost just icing on the cake.
As with running scheduled tests, there is some setup required to offload your tests to another computer. The good news here is that it’s almost exactly what you had to do when you started running scheduled tests: You must install Test Studio Services (TSS) on the computer you want to run tests on. You could be delegating tests to some other computer within an hour. (Download relevant files for this blog post here.)
Setting Up Execution Servers
First, you must pick the computer you’ll use as your execution server and download the Progress Telerik Test Studio Run-time (a separate download from your Telerik account) onto that computer.
When you run that package, don’t install the Storage Services, the Scheduling Server and the Executive Dashboard options—just install the rest of the components in the wizard. If you’ve been running scheduled tests on your Test Studio computer, you’ve already got a scheduling server, storage and the Executive Dashboard installed on your Test Studio computer.
Your next step is to connect your execution server to the scheduling server already running on your Test Studio computer. The URL for your Test Studio computer’s scheduling server is probably http://<name of your Test Studio computer>:8009. If you want to check that, in your Test Studio computer’s Windows menu, type “Configure Test Studio Services” and press the Enter key to open the Test Studio configuration tool. When the Configure Test Services dialog appears, go to the Scheduling tab and you’ll see the Scheduling Service URL there.
The scheduling tab has a convenient Copy URL to Clipboard button you can use if you have some way to transfer this URL to your execution server. While you’re here, also make note of the Storage Server port (probably 8492) right above the scheduling server URL—you may need that port number later to finish connecting your execution server to the Storage Server (the Storage Server is where the execution server will store its copy of the test and where it will report the test’s results).
With that URL and port number in hand, return to your execution server. In that computer’s taskbar, in the Windows notification area on the right, you’ll find the Test Studio icon. Right-click on that icon and select Show from the popup menu to display the Test Studio Test Runner Administrator (this is the tool you use whenever you need to configure your execution server).
To the right of the Scheduling URL entry, click on the builder button (the button with three dots), enter the URL you recorded for your scheduling server, and click the Change button to save your change.
A word of warning here: While computer names are easier to read, if your scheduling server and execution server are in different domains and/or networks, TCP addresses are safer than computer names.
Before you leave the configuration tool, write down the port in the Listening On URL (just below the Scheduling URL in the section called Machine Information)—you’ll need that port number, also (it’s probably 55555, by the way). Unless you have a good reason not to, you should also switch on the “Keep machine awake” toggle in the bottom left corner (in the User Session Configuration section). Setting that to true will keep your execution server from going to sleep when it’s not being used and then being unavailable when you want to run your tests.
Checking Your Connections
You can now set up Test Studio to talk to your execution server. To do that, on your Test Studio computer start Test Studio, open a project and select the menu’s Project tab. In the Project menu’s Scheduling section, click on the Connect icon to open the Scheduling Server Settings dialog (don’t use the Connect icon in the Source Control section).
In the dialog, select the Run remotely option, and click the Connect button in the middle of the dialog. You’re now connected to your Test Studio computer’s scheduling server … which you were before, but with this change, you’ll be able to access your remote execution server from Test Studio. You can tell that you’re connected to your scheduling server because the dialog will display your scheduling server’s logging status.
Close the dialog and, still in Test Studio, go to the Test Lists tab and, in the Execution section, click on the Run List Remotely icon (if you haven’t created a Test List before, you’ll need to use the List icon to create a test list which will enable the Run List Remotely icon).
Clicking the Run List Remotely icon will open a dialog that shows you all your execution servers. You should have at least two: Your Test Studio computer and the execution server you just set up.
If your new execution server isn’t there, it almost certainly means that you need to configure port settings in Windows Defender on both your execution server and your Test Studio Server so that all the servers can talk to each other. Fortunately, that’s not hard to do (and, if your execution server is listed, you can skip the next section).
Configuring Port Settings
There are three ports you need to have open: Two on your Test Studio computer and one on your execution server. Assuming your various servers are running on their default ports (and they probably are), this graphic shows the configuration you want.
Start by configuring the two ports on your Test Studio computer that allow your execution server to communicate with your Test Studio computer’s servers. You recorded the two port numbers you need earlier: the port for your scheduling server (probably 8009) and the port for your storage server where test results are saved (probably 8492).
With those port numbers confirmed, on your Test Studio computer, have Windows Defender open the ports you need.
- In the Windows menu bar, type “Windows Defender” to display the Windows Defender Firewall menu choice. Right-click the menu choice and select Run as Administrator to open Windows Defender Firewall control panel screen.
- On the left, click on the Advanced Settings choice to display the Windows Defender Firewall with Advanced Security dialog.
- In the treeview on the left, click on the Inbound Rules node to display the list of existing rules.
- On the right side, click on the New Rule… link to start the New Inbound Rule Wizard.
- In the wizard, on the Rule Type page, select the Port option and click the Next button.
- On the Protocol and Ports page, make sure the Specific local ports option is selected and enter the port for your scheduling server (probably 8009, but whatever you recorded earlier). Click the Next button.
- On the Action page, make sure Allow the Connection is selected and click the Next button.
- On the Profile page, pick how open you want to leave this port (taking all the defaults is the safest—you can always come back and remove options).
- On the Name page, give your rule a name, add any comments/documentation that seem appropriate and click the Finish button.
Still on your Test Studio computer, run the New Rule wizard again to create a rule that opens the storage server port (probably port 8492 but whatever you recorded earlier).
Now switch to your execution server. Run the Windows Defender Firewall with administrator privileges and repeat the process to create a rule that opens the port that lets your Test Studio computer talk to your execution server. That’s probably port 55555, but use whatever you recorded earlier for the Listening On port when you configured your execution server.
Return to your Test Studio computer, click on the Run Test Remotely icon and the dialog should now show your new execution server (along with your Test Studio computer).
Running Your First Remote Test
With your execution server connected, you can now run a test list on your execution server. After clicking the Run Test Remotely icon to open the dialog that lists your execution servers, check off your new execution server, and click the Run Test List button at the bottom of the dialog. You’ll get a message that says your test is running on the remote computer and you can go check your execution server to see how the test is progressing.
Alternatively, you can wait for the test to finish, go to Test Studio’s Results tab, pick your remote execution from the calendar view, and review your results. But … don’t expect to get the successful run feedback you see here.
Configuring the Execution Server
Your first test run on your execution server probably won’t run successfully because you still need to configure your execution server to support the tests you intend to run on it (and if your first test did run successfully—Congratulations! Skip to the next section).
For example, you’ll need to make sure that the application you want to test is available from your execution server. You may also need to install the Progress Test Studio Extension on the browsers you’ll be using on your execution server. Starting with Test Studio R1 2022, no extension is required if you’re testing in Chrome, which substantially simplifies test setup.
After you’ve done that, calibrating the browsers on your execution server (as you did back when you installed Test Studio) is a good idea and you can do that on the execution server from the TSS Configuration tool in your Windows task bar. You can also save yourself a trip to your execution computer by calibrating the browsers from your Test studio computer’s execution status view or have your browsers calibrated automatically for you in each test run by checking the AutoCalibrateBrowsers option in your Test List settings (the option is on the Web tab behind Edit Settings).
Those requirements apply to every test you’ll run but other configuration requirements will vary from one test to another. If, for example, you’re using an Excel spreadsheet as the data source for a data-driven test and don’t have the Office programs installed on the execution server, you’ll need to make sure that the 2007 Office System Driver is installed on your execution server.
If, when a test fails, the messages on your execution server’s screen don’t give you enough information to determine what you need to change, TSS will let you dig a little deeper. To get some additional information on a failure, open the TSS configuration tool on your execution server from the Task Bar’s notification area. Then, on the Execution tab, use the Logging toggle switch to turn on logging.
Run your test again and, after it completes, use the View Log button to review the results of the test and see what you need to add to your execution server. Once you’ve solved your problem, consider turning off logging until you need it again.
Next Steps
Once you’ve got your test running remotely, you can start running that test remotely on a schedule. If you’ve scheduled tests in the past, there’s not much new here: select the Test Lists menu tab, select the test list you want to schedule and click the Schedule List icon. After entering your schedule and clicking on the Next button, you’ll get the list of all the execution servers that are connected to your Test Studio computer’s scheduling server.
If have multiple execution servers, you can use them in one of two ways:
- If those execution servers each represent a different testing environment, select the execution servers for each environment you want test in. With this choice, all of your tests will be run on every one of the servers you selected and you’ll confirm that your app works in every one of those environments.
- Alternatively, you can use those multiple servers to run your test in parallel and get your testing done faster. For this option, after checking off the servers you want to use, at the bottom of the dialog, check the Distribute tests among these machines checkbox. Each test will now be run on just one of the execution servers you’ve selected, in parallel with tests on other execution servers (the scheduling server will decide what tests to run on which execution server dynamically at run time).
More importantly: Now that you can run your tests on some other computer, you’ve got your Test Studio computer back. You can go back to using it for your real job.
This content originally appeared on Telerik Blogs and was authored by Peter Vogel
Peter Vogel | Sciencx (2022-02-10T09:33:00+00:00) Test Studio Step-by-Step: Running Tests Remotely. Retrieved from https://www.scien.cx/2022/02/10/test-studio-step-by-step-running-tests-remotely/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.