
"Test Automation + Continuous Testing is the holy grail of software development" - Zhimin Zhan


# Description

BuildWise is a lightweight, free and open-source Continuous Testing Server written in Ruby. Apart from being simple, BuildWise has built-in understanding for functional test execution, and distribute test execution on multiple machines. 

For more CI theory, [read this document from Martin Fowler][1].  
You can find documentation and screencasts on BuildWise website [http://agileway.com.au/buildwise][6].

There are several CI/CT solutions already present, why do you need to use BuildWise?

The short answer, just like Martin Flower said in the famous original CI article, many talked about CI, but rare doing it. After frustration with other CI servers, we built one that works for us: running all functional tests multiple times daily, so that we can push update to the production server on green builds. 

Over last decade, our consultants visited many so-called 'Agile' projects, with no exception, their CI processes were all fake, running a handful of tests unreliably at most (and some shamefully fudged test results).  This is not new, back in 2019, renown Agile expert Michael Feathers wrote the same experience in his blog article "UI Test Automation Tools are Snake Oil" (https://cryptochaos.com/ui-test-automation-tools-are-snake-oil):  "It happens over and over again.", when 'agile' project manager got caught on not doing automated end-to-end testing (despite invested tool/resources...),  "we tried.” Michael says, “don’t worry, everyone does."  

Some might acknowledge doing fake Agile in mind, and wanted to ask: "How do we know AgileWay is not faking it with BuildWise?". Seeing is believing, here is our demo BuildWise server https://buildwise.agileway.net/. Yes, we push updates to production server at any time when gets green builds on BuildWise (we trust our tests), and we replicate bugs/failures in automated tests, as a result, we don't use defect tracking at all. 

Main advantages of BuildWise over other solutions include:

1.  BuildWise is very easy to get started, usually in minutes 
2.  BuildWise multi-platform solution: it runs excellent macOS, *nix and Windows.
3.  Parallel test execution: distributed tests to multiple build agents to run them in parallel
4.  Intelligent ordering: most recent failed tests will be given high priority to run first
5.  Auto Retry: A failed test will be assigned to another build agent to run against, greatly reduce false-alarms and execution reliability
6.  Quick feedback, the test result will appear as long as the test execution finishes, no need to wait for whole suite to complete
7.  Easy view individual test histories, which help find out the cause of newly introduced regression error
8.  Screenshot for failed tests
9.  Team members can invoke test execution directly on browser
10. Real lava-lamp integration via Wemo/TP-Link switches
11. Generate live requirement-traceability matrix in seconds (with user supplied user story Excel file)


*BuildWise won 2nd prize of the 10th prestigious Ruby Award in 2018 (Japan)*


## Requirements

* Ruby v2.6+, Recommend Ruby v3.2 (but not v3.3)
  
  Standard Ruby installation. On Windows, please make sure the DevKit is installed. Refer this guide: https://zhiminzhan.medium.com/10-minute-guide-to-set-up-test-automation-using-selenium-webdriver-with-ruby-a2454ac86e95
  
* Git installed in PATH
  
  Standard Git installation. 
  
  Refer this guide: https://zhiminzhan.medium.com/10-minutes-guide-to-git-version-control-for-testers-f58e059bb5e7

* MySQL 5.7+ (for production use) and create a database named "buildwise_production" 
  `mysqladmin –uroot –p create buildwise_production`

* Target browser and its driver, such as Chrome and chromedriver.

## Install

1. unzip buildwise-VERSION.zip to a folder, such as c:\agileway\buildwise-2.5.0

2. Run `bundle install` from a command window (or terminal on Linux/macOS), in the extracted BuildWise folder. 
   
   *for production mode, uncomment a line in Gemfile to include 'mysql2'.
   
3. edit *config\database.yml*, make sure pointing the right database with correct password
   
   BuildWise supports two databases: Sqlite3 and MySQL. We recommend starting with Sqlite3 (called 'demo' env), once you are happy with it, change to MySQL ('production' env). Except the database, there is little different between 'demo' and 'production', the application is the same.

   production:
	  adapter: mysql2
	  encoding: utf8
	  database: buildwise_production
	  host: 127.0.0.1
	  pool: 10
	  username: root
	  password: YOUR_PASSWORD

On Windows, if some libraries not installed properly, install them manually with `--platform=ruby` flag, such as

`gem install --no-document --platform=ruby ffi`
`gem install --no-document --platform=ruby mysql2`


## Start up server

There are two modes.

* with Bundled libraries
  
  Run `bundle` command (in a terminal or command window) to install required libraries (called Gems in Ruby).
  
  Run `bundle-startup.bat` or `bundle-startup-demo.bat`. *(change to .sh on Unix or macOS)*

  In this mode, BuildWise server will use the Gems (libraries) specified in `Gemfile`. 
  
  Pro:
    - the server starts with extactly specified libraries (and versions)
  
  Cons:  
    - However, for any gems you use in your test scripts, you need to add to the Gemfile (sequential build projects only), delete 'Gemfile.lock' and rerun `bundle` command.
  
  
* no Bundles
  
  Run 'startup-demo.bat' or `startup.bat`.

  In this mode, BuildWise server will use the gems installed on the machine. 

  Pros: 
    - using new gems in Ruby test scripts are not limited to the Gemfile (see above)
  
  Cons:  
    - you will need to manage (install/update) the gem installations.

  
**Access via browser**

Open http://localhost:3618 in browser. *(optional)* Change the port number, default to 3618, in startup commands. 


The default working folder for production mode for Windows is *c:\agileway\.buildwise*, which is also specified in the startup commands.


### Upgrade

The release of BuildWise is replaceable, that is, you can simply unzip to a folder with multiple versions side by side like below.

```
 agilweay  
   +- buildwise-2.3.0
   +- buildwise-2.3.1
```

If you are unhappy with the latest version, just simply use the previous one. *(double click startup.bat under it)*

### Configure Project

click 'New project' and follow the instruction there. Typically, you have a working folder (on BuildWise machine) cloned from a repository, then enter the local folder path there. 

## Features

BuildWise currently supports the following SCM tools: 

  * Git

BuildWise currently supports the following notification systems: 

  * Email
  * Wemo
  * Slack
  * TP-Link Smart Plug

BuildWise currently supports the following build systems: 

  * Rake with RSpec

Test frameworks (any automation frameworks in below sytnax, such as Selenium, PlayWright, ...): 
  
  * RSpec (Ruby)
  * Mocha (JavaScript)
  * Cucumber (Ruby)
  * Pytest (Python)


## License
 
BuildWise server is licensed under dual-license model: 

1. Free GPL license, complete license text is included in the License-GPL-3.0.txt file. 

   You may copy, distribute and modify the software as long as you track changes/dates in source files. Any modifications to or software including GPL-licensed code must also be made available under the GPL along with build & install instructions. 

2. Commercial license

   Only for companies redistributing BuildWise server or bundling BuildWise server as their profit-making packages or services.
   
## Support

BuildWise is provided as it is.  Customers who purchased AgileWay's Wise Coaching service can get mentoring support.


## FAQ


1. *Got error: "Mysql2::Error: Incorrect string value: '\xE2\x99\xA5 fr...' for column 'failure' at"*
    
   The reason: The database is not set set UTF-8, but some characters in test output is UTF.   
   [1]:http://www.martinfowler.com/articles/continuousIntegration.html
  
2. “Too many open files” 
   The reason: The files opened by the buildwise process (or Nginx) exceeds the limit, which is set very low
	Verify: ulimit -a
	The workaround: Change the ulimit		
	https://medium.com/mindful-technology/too-many-open-files-limit-ulimit-on-mac-os-x-add0f1bfddde
 
3. Screenshot not present

   Reason: in your test script, did not define $driver
   
     @driver = $driver = Selenium::WebDriver.for(browser_type, browser_options)
 
   This is because $driver is used buildwise_rspec_formatter, which takes a screenshot on a test failure.
   
   Look AgileTravel example Rakefile to how to use buildwise_rspec_formatter 
  
    RSpec::Core::RakeTask.new("ui_tests:quick") do |t|  
      specs_to_be_executed = buildwise_determine_specs_for_quick_build(specs_for_quick_build, excluded_spec_files, $test_dir);
      buildwise_formatter =  File.join(File.dirname(__FILE__), "buildwise_rspec_formatter.rb")
      t.rspec_opts = "--pattern my_own_custom_order --require #{buildwise_formatter} #{specs_to_be_executed.join(' ')} --order defined"
    end
  
