Van Vugt's dynamiXs

It's been a while that I wrote on NAV and automated tests. In the meanwhile I have been teaching a lot all over Europe, and, yes, advocating test automation in NAV. And what did you do since my last post How-to: Run Standard Tests against Your Code? Did you dare and try? And did you also have the time/guts to continue with it? I know some that did.

In the same meanwhile I succeeded to get MS listen and implement a major request I had posed already some time ago. We'll get to that below as it relates to the topic I would like to talk about here: Test Fixture.

If you have been testing software, and probably you do being in my audience, you know that a major part of your testing effort is getting data prepared to allow you to execute your tests. Having been a tester I dare to claim it is THE major part. Test Fixture is the general term for the setup up your system before you execute your test(s), be it software or hardware.

With respect to NAV test automation I have adopted the three level approach Bas Graaf used in his NAV Tech Days presentation ages ago, which he based on xUnit Patterns:

• Prebuilt Fixture
• Lazy Setup or Shared Fixture
• Fresh Setup or Fresh Fixture

Applying this to NAV:

1. Prebuilt Fixture is the database we will run our tests on.
   Typically for the standard Test Toolkit this is CRONUS, containing all sorts of preset data.
2. Lazy Setup or Shared Fixture is the data created at the start of a related number of tests.
   For those who have been looking into these, this is what the Initialize function takes care of in each test codeunit, and typically is about master, supplemental and setup data.
3. Fresh Setup or Fresh Fixture is data created for a specific test only.
   So this is data added to the shared fixture needed to execute a specific test. This often entails transactional data and custom master data.

When I started the project to get the Dynamics NAV Test Toolkit run on our code as per How-to: Run Standard Tests against Your Code, it was giving this result:

23 % percent of a total of 16.124 NAV 2016 tests where successful. Using a statistical approach, tackling the most occurring issues first, I could raise this to 72 % in less than a week. And mainly by focusing on the lazy setup:

Now how did I achieve this code wise?

By creating one generic Initialize function of our own:

Initialize()
IF NOT IsInitialized THEN BEGIN
  IsInitialized := TRUE;
  N4LibraryLabel.CreateLabels;
  N4LibrarySetup.CreateTNTSetups;
  N4LibrarySetup.SetCompanyInformation;
  N4LibrarySetup.SetWarehouseSetup;
  N4LibrarySetup.SetInventorySetup;
  N4LibraryERM.CreateVATPostingSetups;
  N4LibraryERM.UpdateVATProdPostingGroups;
  N4LibrarySetup.SetPurchaseSetup;
  N4LibrarySetup.CreateSalesSetups;
  N4LibrarySetup.CreateVDESetups;
  N4LibrarySetup.CreateOWSSetups;
  N4LibrarySetup.CreateOWSCommunicationSetups;
  N4LibrarySetup.CreateCBSetups;
  N4LibrarySetup.CreateOWSNumbersSetups;
  N4LibrarySetup.CreateMultiple3SNoSeries;
  N4LibrarySetup.CreateTransactionModes;
  N4LibrarySetup.CreateMultiplePaymentTerms;
  N4LibrarySetup.CreateTermPeriodDefinitions;
  N4LibrarySetup.CreateOWSSelections;
  N4LibrarySetup.CreateReqWorksheetTemplates;
  N4LibrarySetup.CreateDeliverabilityCodes;
  N4LibrarySetup.UpdateItems;
END;

... that I hooked into the local Initialize function of each test codeunits, like in the following example in COD134126 (ERM Reversal VAT Entries):

LOCAL Initialize()
LibrarySetupStorage.Restore;
IF IsInitialized THEN
  EXIT;

//<<<
N4LibraryInitialize.Initialize;
//>>>
LibraryERMCountryData.CreateVATData;
LibraryERMCountryData.UpdateGeneralLedgerSetup;
LibraryERMCountryData.UpdateGeneralPostingSetup;
LibraryERMCountryData.UpdateVATPostingSetup;
IsInitialized := TRUE;
COMMIT;
LibrarySetupStorage.Save(DATABASE::"General Ledger Setup");

What the h*ck, lazy setup? Sho-go-nai as the Japanese say, as I learned form Mark Brummel: Nothing can be done about it. Yes, I had to modify 480 test codeunits (of the total of 628) to get this working! And I told MS that I had to. This could have been prevented by a better design of there Initialize function from the start. And yessss, MS listened and now with NAV 2018 CU3 a redesign has been implemented in the first 100 test codeunits:

LOCAL Initialize()
//<<<
LibraryTestInitialize.OnTestInitialize(CODEUNIT::"ERM Apply Sales/Receivables");
//>>>

LibraryVariableStorage.Clear;
LibrarySetupStorage.Restore;
IF isInitialized THEN
  EXIT;
//<<<
LibraryTestInitialize.OnBeforeTestSuiteInitialize(CODEUNIT::"ERM Apply Sales/Receivables");
//>>>
LibraryERMCountryData.CreateVATData;
LibraryERMCountryData.UpdateAccountInCustomerPostingGroup;
LibraryERMCountryData.UpdateGeneralLedgerSetup;
LibraryERMCountryData.UpdateGeneralPostingSetup;
ModifyGenJnlBatchNoSeries;
isInitialized := TRUE;
COMMIT;

LibrarySetupStorage.Save(DATABASE::"General Ledger Setup");
//<<<
LibraryTestInitialize.OnAfterTestSuiteInitialize(CODEUNIT::"ERM Apply Sales/Receivables");
//>>>

where LibraryTestInitialize is COD132250 (Library - Test Initialize) contains these three event publisher to which we now can subscribe:

[External] [IntegrationEvent] OnTestInitialize(CallerCodeunitID : Integer)

[External] [IntegrationEvent] OnBeforeTestSuiteInitialize(CallerCodeunitID : Integer)

[External] [IntegrationEvent] OnAfterTestSuiteInitialize(CallerCodeunitID : Integer)

So getting your lazy setup done on standard tests is a no-brainer. Another threshold taken to get test automation going.

The other day I wanted to get my local NAV 2018 installation fully working. Fully local, including VSCode. And ... setup an extra database with its own service tier.

In the end it was quite easy, not in the least to some very helpful resources (see below), but I ran into a couple of other issues I needed to get fixed. It might be of help to you. Knowing I will typically forget the details, it surely will to be me in the near future. But take advantage of it, if needed. And if you have other details to share be my guest below.

The following steps are in the order in which I tried to get started with VSCode. Typically some steps could be in changed order.

Steps 1 & 2 - Dynamics NAV 2018 and VSCode

Of course both of these needs to be installed first. Note that the VSCode installation is not part of the Dynamics NAV 2018 installation. Be sure to have the installation option Modern Development Environment selected.

Step 3 - Setup new CRONUS database with own service tier

Just for names sake let's call the database My CRONUS and the service tier MyCRONUS110. Port numbers 7145, 7146, 7147, 7148 and 7149.

Step 4 - Enable the Modern Development Environment

I.e. get VSCode and NAV connected. Follow Erik Hougaard's post to get that done. Thanx, Erik.

Changing the Enable Developer Service Endpoint needs a service tier restart, but this failed.

Step 5 - Getting the Developer Service Endpoint URL registered

A service tier failing to start often has to do with the fact the either (1) the service account is not dbowner on the datbase it is servicing or (2) the URL's for the web services ports have not been registered. As the service tier had been running with no problem before I enabled the Developer Service Endpoint it probably needed the URL registered for Developer Service Endpoint: For this Arend-Jan Kauffmann's August 28, 2014 post did come to rescue me, as so many times before.

netsh http show urlacl

... did show it was not registered. So I executed the following command:

http add urlacl url=http://+:7149/MyCRONUS110/ user= "NT AUTHORITY\NETWORK SERVICE" listen=yes delegate=no sddl="D:(A;;GX;;;NS)"

Yoohoo, the service did start.

Note

I am not fully sure, but it seems that developer service endpoint port URL is often not automatically registered.

Step 6 - Open VSCode, create new AL project and run

Yesss, I opened VSCode and did create a new AL project using our future spell AL: Go!

One of the first things to do is setup the launch.json. BTW: calling AL: Go! will urge you to fill out a number of things in the launch.json. A very useful help is another post by Arend-Jan Kauffmann, on setting the

• port number in the launch.json, and
• web client URL on the service tier

Web client URL? Did I have that? Nope, because I did a standard installation and did no have one setup for my new database My CRONUS and service tier MyCRONUS110. So, let's ...

Step 7 - Setup Web Server Instance for the Web Client

Based on this msdn post I created a new web server instance with

New-NAVWebServerInstance -WebServerInstance MyCRONUS110 -Server MyLaptop -ServerInstance MyCRONUS110

And to make sure it accesses the right client port I updated the Microsoft.Dynamics.Nav.Client.ClientService.dll.config:

<add key="ClientServicesPort" value="7146" />

and the navsettings.json:

"ClientServicesPort": 7146

"ManagementServicesPort": 7145

"UnknownSpnHint": "(net.tcp://localhost:7146/MyCRONUS110/Service)=NoSpn"

Well, I didn't just do it like that straight away, to be honest. I did one after the other, in hindsight getting closer and closer to the solution. With each of these changes I restarted the IIS service.

Step 8 - Run the project by means of Ctrl+F5

... and it worked. The web client opened up showing, when click on the Customers link the message as programmed in the HelloWorld.al on the OnOpenPage trigger on Page 22.

Note

With a standard (demo) installation, that includes the Web Server Components and the Modern Development Environment options, many of the above is already setup.

The other day I had to update the objects I use in my training, from 2017 and 2018. Not a lot altogether, thus easy to do this manually and have a look at the changes.

I was glad to have to do this as it showed that a number of fields have been stretched. Fields we typically use in our solution, be it the standard fields themselves or clones of them on our own objects.

So next to the code merge from 2016 to 2018 we were starting on, we were to update our code according to these findings.

You might recall my post on similar changes in NAV 2013, coming from NAV 2009 R2, showing over 450 fields that had been stretched. This time it seems to even be a bigger operation. The current status of my research show me over 800 fields that have been stretched. Mainly code fields stretched from 10 to 20. The following gives a consolidated overview of what I could categorize some way.

Note that the Type of Field column contains either the exact field name (e.g. Bank Account Code) or a kind of consolidated name (e.g. Posting Group Code as there are various fields with different names that refer to a posting group). For the complete overview and details download this Excel file. You will find on the fields tab some 44 more stretched fields that I didn't categorize yet.

Let's confess before I start: getting standard tests to run against your code is by no means rocket science. However, I dare say many peers out there haven't even touched the Test Toolkit, seemingly looking at it as an unbearable threshold. Too much work. Of no relevance to their code. Unknown = unloved (as my mother used to say).

Clearly the majority of the audience I have been speaking to on the Test Automation Suite in the last year and a half, never laid hands on it. I hope I did trigger some of them though, to get started. And if they, and you, still have this cold water fear to get started … this post will show you there's no reason not to. While showing how easy it is, I will also touch on some backgrounds; of the Test Toolkit and test automation in general.

Prerequisites

Now what do we need?

1. A clean CRONUS database serviced on a SQL Server
2. A full NAV installation with DE (development environment) and Windows Clients and Service Tier connecting to the CRONUS database
3. Developers license to run your service tier and DE with
4. Test Toolkit .fob files, to be found in the TestToolKit folder on the product DVD
   • CALTestCodeunits.<country>.fob, containing all test codeunits
   • CALTestLibraries.<country>.fob, containing all generic library codeunits used by the various tests
   • CALTestRunner.fob, containing the test tool framework; note that this already resides in CRONUS so actually no need of this
5. .fob with your app code

Needless to mention that all resources should be based on the same version. In our case we are still running on NAV 2016: code on RTM (build 42815), executables on CU11 (build 46773). And BTW: no direct need for any fancy hardware, virtualized or not. My first steps were done on my laptop!

Baseline

O yes, this CRONUS database. Why are we using a CRONUS database? Probably like you, many participants in my sessions wondered. Why not run it on our own database, containing our data: setup, master data, ledger entries, …?

One of the principles in testing is to have a common baseline when (re)executing tests. To give the best guarantee that a test is reproducible. A in, B out. Every time again. No data created in between two runs of the same test (set) that results in a different outcome. Well, that's not a ground vast argument for using CRONUS, right?

However, another principle is that each test (set) creates its own data needed during execution. And ideally the full Monty, meaning from scratch. Every time again, making a test (set) independent of any data already residing in the system … ideally.

As with many things in live it's not always as ideal as we would like, or as we say. This also applies to the Test Automation Suite.

Yes, the majority of the tests create their own data on the fly, like customers, sales documents, ledger entries, etc. But some also rely on the data readily available in CRONUS, like Sales & Receivables Setup or VAT Posting Setup, data - configuration data - that is used in most relevant tests as-is. I guess this also relates to the fact that the Test Automation Suite is mainly executing integration tests. Thus needing to use an complex set of data. To avoid to have to recreate each part of this set with each execution of a test (set), I guess, MS has chosen to make use of some data readily available in CRONUS and making the test execution faster.

So on one hand, having test create their own data, it is of no importance in what database we execute them. On the other, as described, there is some dependency on data in CRONUS and so we are using CRONUS for our tests.

Note that …

• … we also ran the test on a copy of our own database, but this seemed a never ending exercise; it took ages for the first tests to complete, which seemed related to the huge number of item ledger entries in our database; however, we did never perform a deep analysis on the cause of this as the above shows this is actually not relevant.
• … the Test Automation Suite has been setup in such a way that after the execution of a test codeunit (a test set!) NAV automatically rolls back all changes made to the database. In this way each test codeunit starts o the same data baseline.
• ... on Jan Hoek's request I did elaborate on this here and here.

Enough about this baseline, for now …

Ready … GO!

1. Connect to this CRONUS database with your DE
2. Import the first two Test Toolkit .fob files as mentioned above
3. Import the .fob with your app code
4. Recompile all objects to be sure that everything is OK
   In our case not everything appeared to be OK, due to
   • Missing .dll's
   • Calls to customized standard functions by standard test code
5. If not OK, fix the issues
   BTW: this wasn't a big task to do, but of course might be totally different in your own case.
6. Open Windows Client and open the Test Tool (Departments/Administration/Application Tools)
7. Assuming you have never used this feature in this CRONUS database it should show like this:
   
   A worksheet page with no records, which in this case means that no test codeunits have been added to the DEFAULT test suite
8. To add test codeunits to the DEFAULT test suite click on the Get Test Codeunits action:
   
   As we want to run all available test, select the 2nd option, All Test Codeunits, and click OK.
9. Now all test codeunits (and functions) will be added to the DEFAULT test suite. This might take some time, but once ready …
10. GO, by clicking on the Run action, …
    
    … selecting the 3rd option, All, and pressing OK.

We're in business!

Or at least the test tool should be executing the first test codeunit, 134000 - ERM Apply Sales/Receivables. And of course continue with all the others. In our case this very first run took approx. 1 hour. Which is quite fast as I had ran the standard Test Suite already on standard CRONUS taking approx. 2,5 hours. That it only took an hour was directly due to the fact that only a minority, 23%, of the tests passed and therefor the execution of all tests was much shorter.

But more later, on the results and how we went about getting the failures succeed.

That's it for now and next time don't tell me you still haven't tried to "Run Standard Tests against Your Code".

With our upgrade to NAV 2016 from NAV 2009 R2 many new things came available to us at The Learning Network, formerly known as Van Dijk Educatie. Technical improvements like the performance of the service tier, which was one of the major reasons for wanting to leave behind NAV 2009. But of course, next to a vast number of functional features, also the availability of PowerShell cmdlets, upgrade codeunits and test automation. Those who have been following me one way or another know that the latter subject has my special attention: testing and test automation.

• My series on Test-Driven Development and the testability framework
• The presentations and webinars on the Test Automation Suite for NAV Skills on July 13, 2016 and May 2, 2017, QBS on July 7, 2016, NAVUG on May 10, 2016 and April 6, 2017, Dutch Dynamics Community on January 12, 2016 and June 13, 2017
• The Q&A Jan Hoek initiated
• The upcoming workshops ( Getting Microsoft automated tests working on your solution and Writing your own automated test ) and session on NAV TechDays 2017

In today's post I would like to make a start in sharing our approach and findings in how we started to use the Test Toolkit as being released by MS on each NAV product DVD since NAV 2016. For that I have setup a kind of framework below that allows me refer back to some of its parts when elaborating on it more in posts to come.

So here we go, fasten your seatbelts and stay tuned. And … do not be afraid to ask.

Goal

Our primary goal of this exercise was to setup a test collateral, based on the standard MS application tests, to be used as a regression test suite, by

• Getting as many standard application tests running on our "solution"
• Running these, successful tests against the latest version of our stable code, on a daily basis
• Noting if any of the, formerly, successful tests, have failed (or not, preferably ;-)
• Analyzing failures and fixing them, in either app or test code

Basic Plan

The basic plan to achieve this was to:

1. Run standard tests against our stable code (read more)
2. Select all successful tests
3. Export these as one suite
4. Import as a new, separate suite
5. Run this suite against our stable code
6. Get more standard test running successfully to get more coverage
7. Repeat steps 1 to 7, until all standard tests, or a reasonable, run successful

Assumptions

With any endeavor there are always a number of assumptions. Or should I say loads?

Well, basically we had these two, being that all MS tests …

1. … run successful in CRONUS
   Proved to be nearly true. Approx. 1 ‰ failed; well, also automated tests is software ;-)
2. … are independent, so any test function in any codeunit can be run independent from the other tests in the same test codeunit
   Proved to be true for most of the test functions, but not for all (unfortunately)

Basic Environment

This is the world we live in at The Learning Network:

• App: NAV 2016 NL - RTM (technically CU11)
• Database: CRONUS Nederland BV
• Tests: NAV 2016 NL - RTM, 16.128 tests
• Customization: Ca. 400 standard/Ca. 630 own

To continued ...