Bitcoin Core: Unit Test Failure: system_tests.cpp: Line 48 BOOST_AUTO_TEST(run_command)

Question

I cloned the Bitcoin Core and did the build. But, I am experiencing a Unit Test Failure in system_tests.cpp, Line 48 BOOST_AUTO_TEST(run_command).

First, my environment:

1. Hardware: Intel i7-8550U @1.80GHz 2001 Mhz, 4 Cores, 8 Logical Processors, x64.
2. OS: Windows 10 Home
3. Toolchain: Visual Studios 2019 v142, Configuration: Release, Platform x64.

The build completes without errors, but I do get the following message:

C:\Users\name\source\repos\bitcoin-first\build_msvc\test_bitcoin\test_bitcoin.vcxproj : warning : Wildcards in project items are not supported in VC projects and can cause unexpected behavior, instability and performance degradation in the Visual Studio IDE. Please refer to https://aka.ms/cpp/projectwildcards for supported options.

Running test_bitcoin, all tests pass except run_command from source system_tests.cpp line 48. I get the following error:

struct boost::process::process_error: CreateProcess failed: The system cannot find the file specified.

VS Test Explorer gives additionally:

unknown location(0): exception in "run_command": struct boost::process::process_error: CreateProcess failed: The system cannot find the file specified. system_tests.cpp(52): last checkpoint:

The relevant code from system_tests.cpp (Please disregard any spacing syntax issues after the line numbers. I manually added them in and they are not at issue):

48 BOOST_AUTO_TEST_CASE(run_command)
49 {
50    {
51        const UniValue result = RunCommandParseJSON("");
52        BOOST_CHECK(result.isNull());
53    }
54    {
55 #ifdef WIN32
56       // Windows requires single quotes to prevent escaping double quotes from the JSON...
57       // const UniValue result = RunCommandParseJSON("cmd.exe /c echo '{\"success\": true}'");
58        const UniValue result = RunCommandParseJSON("echo '{\"success\": true}'");
59   
60 #else
61        // ... but Linux and macOS echo a single quote if it's used
62        const UniValue result = RunCommandParseJSON("echo \"{\"success\": true}\"");
63 #endif
64        BOOST_CHECK(result.isObject());
65        const UniValue& success = find_value(result, "success");
66        BOOST_CHECK(!success.isNull());
67        BOOST_CHECK_EQUAL(success.getBool(), true);
68    }
69    {
70        // An invalid command is handled by Boost
71        BOOST_CHECK_EXCEPTION(RunCommandParseJSON("invalid_command"), boost::process::process_error, checkMessage); // Command failed
72    }
73    {
74        // Return non-zero exit code, no output to stderr
75        BOOST_CHECK_EXCEPTION(RunCommandParseJSON("false"), std::runtime_error, checkMessageFalse);
76    }
77    {
78        // Return non-zero exit code, with error message for stderr
79        BOOST_CHECK_EXCEPTION(RunCommandParseJSON("ls nosuchfile"), std::runtime_error, checkMessageStdErr);
80    }
81    {
82        BOOST_REQUIRE_THROW(RunCommandParseJSON("echo \"{\""), std::runtime_error); // Unable to parse JSON
83    }
84    // Test std::in, except for Windows
85 #ifndef WIN32
86    {
87        const UniValue result = RunCommandParseJSON("cat", "{\"success\": true}");
88        BOOST_CHECK(result.isObject());
89        const UniValue& success = find_value(result, "success");
90        BOOST_CHECK(!success.isNull());
91        BOOST_CHECK_EQUAL(success.getBool(), true);
92    }
93 #endif
94 }
95 #endif // HAVE_BOOST_PROCESS

Line 58 is failing. Based on this https://stackoverflow.com/questions/47028660/boost-createprocess-failed, I commented out line 58 and added the following Line 57:

57  const UniValue result = RunCommandParseJSON("cmd.exe /c echo '{\"success\": true}'");

Running again the system_tests.cpp gets me past the CreateProcess failure, but produces the following failure in system_tests.cpp line 48, run_command():

class std::runtime_error: Unable to parse JSON: '{"success": true}'.

VS Test Explorer additionally states:

unknown location(0): exception in "run_command": class std::runtime_error: Unable to parse JSON: '{"success": true}' system_tests.cpp(52): last checkpoint:

I'm having a hard time believing the test is incorrect--I've found nothing about this failure. Instead, I suspect I didn’t correctly set some switch or option or configuration somewhere.

Any comments or help would be greatly appreciated.

Answer 1

Here is my work around. I still have a hard time believing that the test code needs revision, but this is what I did and the tests now pass.

From the above, I changed the call of RunCommandParseJSON() to provide for the explicit call of the command window. Apparently, this is needed for Windows. The line became:

57 const UniValue result = RunCommandParseJSON("cmd.exe /c echo '{\"success\": true}'");

RunCommandParseJSON() is called from /src/util/system.cpp. It takes a string input representing a child process, which it is to start. In this case, start the command window and echo the string '{"success": true}'.

This produced a runtime error:

class std::runtime_error: Unable to parse JSON: '{"success": true}'

Note: the string output from the echo command begins and ends with a single quote. As noted in the original comments to the code, single quotes are needed to avoid Windows escaping the double quotes from the JSON.

RunCommandParseJSON() calls UniValue::read() from /src/univalue/lib/univalue_read.cpp. UniValue::read(), does not seem to accommodate the case of a string input beginning and ending with single quotes.

Thus, I changed line 59 from:

while (raw < end && (json_isspace(*raw))) // skip whitespace

to:

while (raw < end && (json_isspace(*raw) || (*raw == '\''))) // skip whitespace and skip single quote to begin and end a json string

This allowed the for the

const UniValue result = RunCommandParseJSON("cmd.exe /c echo '{\"success\": true}'");

aspect of the run_command test to succeed.

Still, I was getting errors on run_command when further testing for throwing the proper errors:

1. No output to stderr

2. Error message to stderr

3. Unable to parse JSON error

4. The no output to stderr case tests for when you can start the command window process, but the command given is invalid. This test was revised to allow for the proper calling of the Window’s command window, changing from:

BOOST_CHECK_EXCEPTION(RunCommandParseJSON("false"), std::runtime_error, checkMessageFalse);

to:

BOOST_CHECK_EXCEPTION(RunCommandParseJSON("cmd.exe /c false"), std::runtime_error, checkMessageFalse);

This produced the desired error, but not the exact error message called for by the test. I changed the check to:

const std::string what(ex.what()); BOOST_CHECK(what.find("returned 1") != std::string::npos);

2. The Error Message to Stderr tests for being able to start the command window process, receive a valid command, but invalid input. This was changed from:

BOOST_CHECK_EXCEPTION(RunCommandParseJSON("ls nosuchfile"), std::runtime_error, checkMessageStdErr);

to:

BOOST_CHECK_EXCEPTION(RunCommandParseJSON("cmd.exe /c dir nosuchfile"), std::runtime_error, checkMessageStdErr);

to replicate what I see as the Window’s version of the test.

3. The Unable to ParseJSON error tests for ability to start the command window process, send a valid command with valid input, but the string isn’t able to be parsed. (This error was actually encountered early on.) This was changed to:

BOOST_REQUIRE_THROW(RunCommandParseJSON("cmd.exe /c echo '{'"), std::runtime_error); // Unable to parse JSON

With these changes all unit tests now pass.

I don’t see that these changes changed the nature of the tests. If someone notices that they do, I would definitely appreciate being told.

Answer 2

Perhaps the following will work.

RunCommandParseJSON("cmd.exe", "/c echo '{\"success\": true}'");