How to run Testtoolkit in Azure DevOps

Hello,

I've started to follow a couple of blogs and GitHub repositories around the topic testing for AL in Azure DevOps. I've implemented a pipeline which ultimately executes my own tests automatically and returns the result to Azure DevOps where it's displayed nicely.

Now I received the task to also run the existing Microsoft tests to see if our apps somehow messed up some default functionality, but this seems to be complicated. I got 2 specific questions to kick the discussion off:

  • Should it be possible to run all Microsoft Tests which my apps installed without UI?
  • How can I identify the codeunits and methods affected by my apps, so I can run those tests specifically.

If I missed some crucial information please let me know and I'll try to fill the voids. I started using this video 

https://www.youtube.com/watch?v=1XCJzRF3c8U

Thank you for your help

  • Should it be possible to run all Microsoft Tests which my apps installed without UI?

    No, there are quite a number of standard tests that need a client session running (in the background)

    How can I identify the codeunits and methods affected by my apps, so I can run those tests specifically.

    First make a full test run with all your test codeunits included. Have the property Update Test Coverage Map turned on on your test suite. Drilldown on Hit Objects will show you what is hit.

  • Hey Luc,

    thanks for your reply. Regarding the client session I found this blog and this youtube video from Jasminka Thunes today and from my understanding both suggest it should be possible to run the test codeunits from a console session.

    Regarding the coverage map: If the information is included in the coverage map I should be able to export it and examin it programmatically too right?

    After posting my question yesterday, I found that using microsoft/bcsandbox:base image improved my results on Azure DevOps dramatically and out of 980 tests I executed as a first try only 1 failed.

  • it should be possible to run the test codeunits from a console session.

    Yes, as long as they do not have some UI elements used like TestPages and Reports.

    should be able to export it and examin it programmatically too right?

    I guess so, but haven't done anything with that yet. Based on the coverage and items selected using the Get Test Codeunits action you can generate a new test suite. That you can surely export.

  • Hi Andreas,

    Luc is way to humble to even suggest his own blog Van Vugt's dynamiXs and videos.