[taler-docs] branch master updated: integration testing and fault injection write-up

[gnunet]

This is an automated email from the git hooks/post-receive script.

dold pushed a commit to branch master
in repository docs.

The following commit(s) were added to refs/heads/master by this push:
     new 8302beb  integration testing and fault injection write-up
8302beb is described below

commit 8302beb9ef13811287e16aa27109821b6f660ce0
Author: Florian Dold <florian.dold@gmail.com>
AuthorDate: Thu Aug 6 15:07:48 2020 +0530

    integration testing and fault injection write-up
---
 taler-wallet.rst | 122 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 122 insertions(+)

diff --git a/taler-wallet.rst b/taler-wallet.rst
index a4e65f6..ae0656d 100644
--- a/taler-wallet.rst
+++ b/taler-wallet.rst
@@ -921,3 +921,125 @@ There are test cases that require us to modify the 
communication between the wal
 * What does the wallet do when the exchange/merchant announce an incompatible 
protocol version?
 * What happens if some signature made by the exchange/merchant is garbage?
 * What if the exchange reports a double-spend and the proof it gives us is 
invalid?
+
+
+
+Integration Test and Fault Injection Framework
+----------------------------------------------
+
+This section describes the current approach to integration testing in the 
wallet.
+
+It's all based on a TypeScript harness process, which itself implements
+the fault injection proxy (async and in-process)!
+
+The new approach consists of the following parts:
+
+1. A strongly typed, convenient helper library to easily set up and run
+arbitrary Taler deployments and run test cases.  These components plug
+together as easily as lego bricks, even with multiple
+exchanges/merchants/banks/etc.  Logs and clean shutdown (even on SIGINT
+or errors) are handled properly.  (Support for auditors is still pending
+but needed to fully test the wallet.)
+
+This is how a simple withdrawal and payment test case looks like:
+`<https://git.taler.net/wallet-core.git/tree/packages/taler-integrationtests/src/test-payment.ts>`__
+
+(What's particularly nice is that all our docs contain TypeScript
+definitions for all API request bodies.  So just copying them into the
+test harness gives us auto-completion and compile-time checks to avoid
+typos.  The wallet's JSON validation machinery is also re-used.)
+
+2. A fault injection proxy that can be plugged between the services
+and/or the wallet.  It runs alongside the test harness, and can thus can
+use arbitrary custom logic.  There's no dependency for it other than
+built-in Node.JS libraries.  Simple fault injections are just as easy to
+set up as with the twister.
+
+The following test case (a) logs all requests and responses to the test
+harness stdout and (b) at a certain point, starts dropping the next 10
+requests to the exchange (testing the wallet's retry logic):
+
+`<https://git.taler.net/wallet-core.git/tree/packages/taler-integrationtests/src/test-payment-fault.ts#n165>`__
+
+3. All util functionality from JS wallet-core, such as the Taler crypto,
+amount/date/etc. handling and JSON parsing/validation (the wallet is now
+more modular and easier to use as a library) can be used in the
+integration tests, even if a different wallet (Kotlin, whatever) is
+tested via the CLI.
+
+4. A bunch of test cases that use (1)-(3).  These are *significantly*
+more readable and hackable than other test approaches we had, while
+allowing for more complex scenarios.  There are still way too few tests
+though!
+
+5. A test runner (written in bash) that runs test cases based on a glob
+pattern and reports the results.
+
+Injecting a fault is as easy as:
+
+.. code:: ts
+
+  // Set up test case
+  [...]
+
+  exchangeProxy.addFault({
+    beforeResponse(ctx: FaultInjectionResponseContext) {
+       if (cond1) { // Drop some responses
+         ctx.dropResponse = true;
+         return;
+       } else if (cond2) { // modify some others
+         ctx.responseBody = Buffer.from(`{"oops": true}`, "utf-8");
+         return;
+       }
+       // Other things that can be modified:
+       // - drop/modify the request, not just the response
+       // - modify headers
+       // - modify status codes
+    }
+  });
+
+  await doSomethingWithTheWallet();
+
+  exchangeProxy.clearFault();
+
+  await doMoreWithTheWallet();
+
+
+To make the configuration easy, an ExchangeService (or MerchantService,
+BankService etc.) can be wrapped in a FaultInjectedExchangeService,
+which implements the ExchangeServiceInterface:
+
+.. code:: ts
+
+  // create exchange and two merchants
+  const exchange = await setupExchange(...);
+  const merchant1 = ...;
+  const merchant2 = ...;
+
+  // Add exchange to merchant-accepted exchanges.
+  // This will adjust the config.
+  merchant1.addExchange(exchange);
+
+  // Wrap exchange in fault injection proxy
+  const faultInjectedExchange: ExchangeServiceInterface
+    = new FaultInjectedExchangeService(t, exchange1, 8085);
+
+  // Merchant 2 talks to the exchange over fault injection,
+  // and thus must use the "twisted" base URL.
+  merchant2.addExchange(faultInjectedExchange);
+
+
+The package for the integration tests is here:
+
+`<https://git.taler.net/wallet-core.git/tree/packages/taler-integrationtests>`__
+
+The shortcut to run all integration tests is
+
+.. code:: sh
+
+  ./bootstrap && ./configure --prefix=... \
+     && make install integrationtests
+
+(From the root of the whole repo.  If you're developing tests, it's way
+faster to just run "make compile install" once and then use
+"./testrunner" from the taler-integrationtests package.)

-- 
To stop receiving notification emails like this one, please contact
gnunet@gnunet.org.