Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

various updates (#23) #59

Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
137 commits
Select commit Hold shift + click to select a range
ec9b1b1
Fix Cucumber parallel formatter
Jan 25, 2018
a41828b
Fix parallel_tests version and possible endless loop
Jan 29, 2018
97e1f5d
Unify common code in Cucumber Formatter and ParallelFormatter
Jan 30, 2018
69da60c
Add debug logging to debug some issue
Jan 30, 2018
75ad0f9
Set report_on_exception = true for Report Portal thread on Ruby 2.4+
Jan 30, 2018
3def39e
Add more debug logging
Jan 30, 2018
25c5159
Remove retrying of requests to Report Portal
Jan 30, 2018
f18bb5e
Set last_used_time to a time of the parent item
Jan 30, 2018
396a961
Remove log debug statements
Jan 31, 2018
e1c4805
Set description to cmd args of parallel_tests process
Feb 1, 2018
623fd05
Pin parallel_tests Gem to 2.15.0 and sys-proctable Gem to 1.1.5
Feb 22, 2019
80c4380
Fix URLs to include required "filter.eq.launch" parameter
Feb 22, 2019
1153181
Fix matcher to detect when start time needs to be updated
Feb 22, 2019
05b83ba
Clean up formatting in report.rb
Feb 22, 2019
62b388d
Implement a folder creation lock file to address duplicate folders be…
Feb 22, 2019
4ea36e6
Delay each process by its ID number to create a staggered start to re…
Feb 22, 2019
3c58d20
Formatting and cleanup in parallel_report.rb
Feb 22, 2019
e11290b
Merge branch 'fix_parallel_cucumber_formatter'
Feb 22, 2019
580850b
Use temporary directory for tracking folder creation
Mar 19, 2019
b188a34
Add commenting around why we are delaying threads
Mar 19, 2019
10fb831
Make sure to use a "full path" for folder creation tracking
Mar 19, 2019
2c26a48
Merge pull request #1 from mikesalvia/master
vveliev-tc Apr 11, 2019
357e707
Switch to use class instance variables instead of global variable for…
Apr 12, 2019
868f789
Merge pull request #33 from mikesalvia/master
Apr 13, 2019
c540c01
work in progress, refactoring lock generation for parallel tests
vveliev-tc May 24, 2019
7ed6e2c
fixing parallel execution
vveliev-tc May 25, 2019
c49d5eb
Merge branch 'fix_parallel_cucumber_formatter' into feature/cucumber_…
vveliev-tc May 25, 2019
a69014c
Merge pull request #2 from vveliev-tc/feature/cucumber_parallel
vveliev-tc May 25, 2019
c01819c
small refactoring
vveliev-tc May 27, 2019
86bc1c7
Merge pull request #3 from vveliev-tc/feature/cucumber_parallel
vveliev-tc May 27, 2019
ecc4801
fixing report description when run in parallel
vveliev-tc May 27, 2019
e62541e
fixing report description when run in parallel
vveliev-tc May 27, 2019
01fbe3a
- removing debugger
vveliev-tc May 31, 2019
c15caef
- reverting file lock changes for reading the file
vveliev-tc May 31, 2019
494fda0
Merge pull request #4 from vveliev-tc/feature/cucumber_parallel
vveliev-tc Jun 20, 2019
dad6361
- fixing temp file path generation
vveliev-tc Jun 20, 2019
9f2bbde
Merge pull request #5 from vveliev-tc/hotfix/parallel_runner
vveliev-tc Jun 20, 2019
a76e451
Merge pull request #6 from vveliev-tc/master
vveliev-tc Jun 20, 2019
e40548b
- reverting some changes
vveliev-tc Jun 20, 2019
1e2628e
Merge pull request #7 from vveliev-tc/hotfix/parallel_cucumber
vveliev-tc Jun 20, 2019
624cd59
Merge branch 'master' into feature/cucumber_parallel
vveliev-tc Jun 20, 2019
ad05252
Fix Cucumber parallel formatter
Jan 25, 2018
5067695
Fix parallel_tests version and possible endless loop
Jan 29, 2018
3242ac1
Add debug logging to debug some issue
Jan 30, 2018
ab9483c
Add more debug logging
Jan 30, 2018
c25fb0e
Remove retrying of requests to Report Portal
Jan 30, 2018
4fbf863
Set last_used_time to a time of the parent item
Jan 30, 2018
41e0604
Remove log debug statements
Jan 31, 2018
24e16f6
Set description to cmd args of parallel_tests process
Feb 1, 2018
1ec43e8
Fix URLs to include required "filter.eq.launch" parameter
Feb 22, 2019
d1c128a
Fix matcher to detect when start time needs to be updated
Feb 22, 2019
b62c7eb
Clean up formatting in report.rb
Feb 22, 2019
ad7c8d6
Implement a folder creation lock file to address duplicate folders be…
Feb 22, 2019
b9b6b14
Delay each process by its ID number to create a staggered start to re…
Feb 22, 2019
1f40f58
Formatting and cleanup in parallel_report.rb
Feb 22, 2019
e61d8f7
Use temporary directory for tracking folder creation
Mar 19, 2019
3bc8d9a
Add commenting around why we are delaying threads
Mar 19, 2019
893af0f
Make sure to use a "full path" for folder creation tracking
Mar 19, 2019
a92fc75
Switch to use class instance variables instead of global variable for…
Apr 12, 2019
c6bb6ec
work in progress, refactoring lock generation for parallel tests
vveliev-tc May 24, 2019
f97e072
fixing parallel execution
vveliev-tc May 25, 2019
d2d2cbe
small refactoring
vveliev-tc May 27, 2019
f30977b
fixing report description when run in parallel
vveliev-tc May 27, 2019
265edef
- removing debugger
vveliev-tc May 31, 2019
0d3fba8
- reverting file lock changes for reading the file
vveliev-tc May 31, 2019
6215a53
- fixing temp file path generation
vveliev-tc Jun 20, 2019
3e655b6
- reverting some changes
vveliev-tc Jun 20, 2019
d31ab44
Merge branch 'master-upstream' into feature/cucumber_parallel
vveliev-tc Jun 20, 2019
8985eca
- fixes after merge
vveliev-tc Jun 21, 2019
f5cc434
layout fixes
vveliev-tc Jun 21, 2019
0cb6a6c
layout fixes
vveliev-tc Jun 21, 2019
9056c74
layout fixes
vveliev-tc Jun 21, 2019
52707ca
layout fixes
vveliev-tc Jun 21, 2019
b3359a7
layout fixes
vveliev-tc Jun 21, 2019
af18470
Merge branch 'master' into feature/cucumber_parallel
vveliev-tc Jun 21, 2019
fe8d51d
fixing report completion step
vveliev-tc Jun 24, 2019
aabebcf
Merge remote-tracking branch 'upstream/skip-reporting-hierarchy' into…
vveliev-tc Jun 24, 2019
7694f48
fixes after merge
vveliev-tc Jun 24, 2019
37fbb35
- fixing retry error
vveliev-tc Jun 25, 2019
e132c1c
Merge pull request #9 from vveliev-tc/feature/cucumber_parallel
vveliev-tc Jun 25, 2019
f338909
minor layout fixes
vveliev-tc Jun 25, 2019
58d0468
Merge remote-tracking branch 'upstream/master' into feature/cucumber_…
vveliev-tc Jun 25, 2019
b929ce8
minor layout fixes
vveliev-tc Jun 25, 2019
1cb735e
Merge pull request #10 from vveliev-tc/feature/cucumber_parallel
vveliev-tc Jun 25, 2019
a6edfa7
using single formatter for both parallel_cucumber and cucumber
vveliev-tc Jun 25, 2019
aa0342c
using single formatter for both parallel_cucumber and cucumber
vveliev-tc Jun 25, 2019
8549179
adding logger for report portal
vveliev-tc Jun 27, 2019
0da36cc
updating readme and adding settings for logging level
vveliev-tc Jun 27, 2019
f668bfb
Merge pull request #11 from vveliev-tc/feature/agent_logger
vveliev-tc Jun 27, 2019
321f256
updating how response processing is handling
vveliev-tc Jun 27, 2019
9ba5fce
removing debugger
vveliev-tc Jun 27, 2019
10af2f4
Merge pull request #12 from vveliev-tc/feature/response_processing
vveliev-tc Jun 27, 2019
92671e7
fixing issue with child process not closing the folders items
vveliev-tc Jun 28, 2019
cad3f7c
Merge pull request #13 from vveliev-tc/feature/response_processing
vveliev-tc Jun 28, 2019
1c3b42f
fixing value for time, removing cucumber IO as it's no longer needed
vveliev-tc Jun 28, 2019
369af40
Merge pull request #14 from vveliev-tc/feature/response_processing
vveliev-tc Jun 28, 2019
8cf2206
- adding logging
vveliev-tc Jun 28, 2019
475eac9
Merge pull request #15 from vveliev-tc/hotfix/finishing_launch
vveliev-tc Jun 28, 2019
e559f3f
- adding logging
vveliev-tc Jun 28, 2019
61c4bbb
Merge pull request #16 from vveliev-tc/hotfix/item_already_closed
vveliev-tc Jun 28, 2019
d631143
Merge remote-tracking branch 'origin/master'
vveliev-tc Jun 29, 2019
2cb3597
- fixing file lock priority
vveliev-tc Jul 2, 2019
3a7c4a2
Merge pull request #17 from vveliev-tc/hotfix/file_lock
vveliev-tc Jul 2, 2019
638a7bf
remove debugger
vveliev-tc Jul 2, 2019
e8d5dbb
Merge pull request #18 from vveliev-tc/hotfix/backtrace
vveliev-tc Jul 2, 2019
a79d207
updating logger message on Error rescue
vveliev-tc Jul 3, 2019
67e1092
Merge pull request #19 from vveliev-tc/hotfix/update_logger
vveliev-tc Jul 3, 2019
45b47a6
adding patch for multipart
vveliev-tc Jul 8, 2019
2794ce9
Merge remote-tracking branch 'origin/master'
vveliev-tc Jul 8, 2019
abb0cb8
fixing multipart post , adding some logs
vveliev-tc Jul 10, 2019
c3a7f79
reverting unwanted changes
vveliev-tc Jul 10, 2019
3c06300
Merge pull request #20 from vveliev-tc/feature/new_http_request_lib
vveliev-tc Jul 10, 2019
66799ab
Merge branch 'master' of github.com:vveliev-tc/agent-ruby
vveliev-tc Jul 10, 2019
083a83a
updating version
vveliev-tc Jul 10, 2019
a64ff9e
various updates (#23)
vveliev-tc Jul 10, 2019
1e50fec
fixing layout errors
vveliev-tc Jul 10, 2019
2340184
Merge branch 'master' into feature/cucumber_parrallel_fariday
vveliev-tc Jul 10, 2019
f8e75c6
adding gemspec dependency
vveliev-tc Jul 11, 2019
1d37021
updating readme
vveliev-tc Jul 11, 2019
9fd2cda
fixing gemspec, code formatting
vveliev-tc Jul 17, 2019
2c447d8
fixing typo, gemspec update
vveliev-tc Jul 17, 2019
7be5353
fixing issue with formatter when file with launch id is provided
vveliev-tc Jul 18, 2019
92af68d
fixing typos
vveliev-tc Aug 9, 2019
72d496a
fixing typos
vveliev-tc Aug 9, 2019
efc07c0
fixing typos
vveliev-tc Aug 9, 2019
d657458
Feature/cucumber parrallel fariday (#24)
vveliev-tc Aug 9, 2019
00e5d88
Merge branch 'master' into develop
vveliev-tc Aug 9, 2019
da82da3
Merge pull request #25 from vveliev-tc/develop
vveliev-tc Aug 9, 2019
cd14bd3
Merge branch 'master' into feature/cucumber_parrallel_fariday
vveliev-tc Aug 16, 2019
0811ede
Merge pull request #26 from vveliev-tc/feature/cucumber_parrallel_far…
vveliev-tc Aug 16, 2019
ead9cc1
splitting client to stand alone class, need to add error handling
vveliev-tc Aug 16, 2019
8414edc
fixing patch typo, making sure attachments work
vveliev-tc Aug 16, 2019
3c76071
Merge branch 'develop' into feture/cleanup
vveliev-tc Aug 16, 2019
69ed4ae
Merge pull request #27 from vveliev-tc/feture/cleanup
vveliev-tc Aug 16, 2019
c7c177b
Merge pull request #28 from vveliev-tc/develop
vveliev-tc Aug 16, 2019
2de8740
rubocop fix
vveliev-tc Aug 16, 2019
e400952
Merge branch 'master' into feature/cucumber_parrallel_fariday
vveliev-tc Sep 4, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Metrics/BlockLength:
# Offense count: 2
# Configuration parameters: CountComments.
Metrics/ClassLength:
Max: 169
Max: 350

# Offense count: 3
Metrics/CyclomaticComplexity:
Expand All @@ -66,7 +66,8 @@ Metrics/MethodLength:
# Offense count: 1
# Configuration parameters: CountComments.
Metrics/ModuleLength:
Max: 162
Max: 250


# Offense count: 5
Metrics/PerceivedComplexity:
Expand Down Expand Up @@ -112,8 +113,6 @@ Style/Documentation:
- 'spec/**/*'
- 'test/**/*'
- 'lib/report_portal/cucumber/formatter.rb'
- 'lib/report_portal/cucumber/parallel_formatter.rb'
- 'lib/report_portal/cucumber/parallel_report.rb'
- 'lib/report_portal/logging/logger.rb'
- 'lib/report_portal/patches/rest_client.rb'
- 'lib/report_portal/rspec/formatter.rb'
Expand Down Expand Up @@ -168,7 +167,6 @@ Style/GlobalVars:
# Configuration parameters: MinBodyLength.
Style/GuardClause:
Exclude:
- 'lib/report_portal/cucumber/parallel_report.rb'
- 'lib/report_portal/cucumber/report.rb'
- 'lib/report_portal/rspec/formatter.rb'
- 'lib/reportportal.rb'
Expand Down
11 changes: 6 additions & 5 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ source 'https://rubygems.org'
gemspec

gem 'cucumber', '~> 3'
gem 'parallel_tests'
gem 'rake'
gem 'rspec'
gem 'rubytree', git: 'https://github.com/razboev/RubyTree'

gem 'faraday'
gem 'log4r'
gem 'logging'
gem 'parallel_tests', '~> 2.15.0'
gem 'rake'
gem 'rspec'
gem 'rubytree'
gem 'sys-proctable', '~> 1.1.5'
58 changes: 42 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,18 @@ Add `gem 'reportportal', git: 'https://github.com/reportportal/agent-ruby.git'`

* With Cucumber:

```cucumber <other options> -f ReportPortal::Cucumber::Formatter```
```cucumber <other options> -f ReportPortal::Cucumber::Formatter -o '<log_file>'```

* With Cucumber and parallel_tests gem:

```parallel_cucumber <some options> -o '<some other options> -f ReportPortal::Cucumber::ParallelFormatter'```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to have an example for parallel_cucumber to show that the gem supports parallel_tests.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're missing the point, it's single formatted for both

Copy link
Member

@abotalov abotalov Aug 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that it's a single formatter. But the potential user looking at README should also be aware that agent works when tests are run in parallel.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are more than welcome to contribute.

* With Cucumber (Advanced)

```ruby
AfterConfiguration do |config|
...
#rp_log_file = <Generated or from ENV>
...
config.formats.push(["ReportPortal::Cucumber::Formatter", {}, rp_log_file])
end
```
* With RSpec:

```rspec <other options> -f ReportPortal::RSpec::Formatter```
Expand All @@ -54,6 +60,8 @@ Supported settings:
- launch_id - id of previously created launch (to be used if formatter_modes contains attach_to_launch)
- file_with_launch_id - path to file with id of launch (to be used if formatter_modes contains attach_to_launch)
- disable_ssl_verification - set to true to disable SSL verification on connect to ReportPortal (potential security hole!). Set `disable_ssl_verification` to `true` if you see the following error:
- launch_uuid - when formatter_mode is `attach_to_launch`, launch_uuid will be used to create uniq report group(tmp dir should be shared across all lunches)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Put a space before the opening parenthesis - report group (tmp dir should be shared across all lunches)
  • lunches -> launches
  • formatter_mode is `attach_to_launch` -> formatter_modes contains `attach_to_launch`

I don't see a difference between launch_id and launch_uuid. launch_id is also a UUID of the launch to be used for attach_to_launch mode. If launch_id is provided by the user, reporting should happen to that launch.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lunch_id is coming from the report portal and forcing you to create lunch beforehand, with luanch_uuid, there is no need to create lunch beforehand. I'm not using the rake and not familiar with it so we have different approaches to solve the same problem.

- log_level - this is log level for report_portal agent (useful for troubleshooting issued when run in parallel mode)
vveliev-tc marked this conversation as resolved.
Show resolved Hide resolved
```
Request to https://rp.epam.com/reportportal-ws/api/v1/pass-team/launch//finish produced an exception: RestClient::SSLCertificateNotVerified: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
```
Expand All @@ -75,12 +83,36 @@ WebMock.disable_net_connect!(:net_http_connect_on_start => true, :allow_localhos
## Formatter modes

The following modes are supported:

| Name | Purpose |
| --- | --- |
| attach_to_launch | Do not create a new launch but add executing features/scenarios to an existing launch. Use launch_id or file_with_launch_id settings to configure that. If they are not present client will check rp_launch_id.tmp in `Dir.tmpdir`)
| use_same_thread_for_reporting | Send reporting commands in the same main thread used for running tests. This mode is useful for debugging this Report Portal client. It changes default behavior to send commands in the separate thread. Default behavior is there not to slow test execution. |
| skip_reporting_hierarchy | Do not create items for folders and feature files |
<table><thead><tr><th>Name</th><th>Purpose</th></tr></thead>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is rendered with horizontal scrolling in Github. So a text becomes harder to read in comparison to the current table rendered using Github flavored markdown.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CapturFiles-20190808_084246

I don't see any scroling

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.github.com/gfm/#html-blocks - not sure what you are asking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you can see on the screenshot you attached, not all text in column "Purpose" is visible. There is a horizontal scrolling (at least in Chrome).
https://github.com/reportportal/agent-ruby/blob/7be5353c31be361a2a73b93b83f8e4dffacd8c1b/README.md
Screenshot from 2019-08-09 11-50-04

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, but I don't see what is the issue here. If you know how to do multiple lines in a simple table block let me know.

Copy link
Member

@abotalov abotalov Aug 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previous formatting looked better in the browser IMO. Could you use it?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is subjective. Put everything in a single paragraph, and it will not look good in the browser IMO. Hense I had to change it to be more presentable.

<tbody>
<tr>
<td>attach_to_launch</td>
<td>
Add executing features/scenarios to same launch.
Use following options are available to configure that.

1. launch_id
2. file_with_launch_id
3. launch_uuid
4. rp_launch_id_for_<first process pid>.lock in `Dir.tmpdir`

*** If launch is not created, cucumber process will create one. Such that user does not need to worry about creating new launch.
</td>
</tr>
<tr>
<td>use_same_thread_for_reporting</td>
<td>
Send reporting commands in the same main thread used for running tests. This mode is useful for debugging
Report Portal client. It changes default behavior to send commands in the separate thread.
Default behavior is there not to slow test execution. </td>
</tr>
<tr>
<td>skip_reporting_hierarchy</td>
<td>
Do not create items for folders and feature files</td>
</tr>
</tbody>
</table>

## Logging
Experimental support for three common logging frameworks was added:
Expand All @@ -91,12 +123,6 @@ Experimental support for three common logging frameworks was added:

To use Logger, set use_standard_logger parameter to true (see Configuration chapter). For the other two corresponding appenders/outputters are available under reportportal/logging.

## Parallel formatter

ReportPortal::Cucumber::ParallelFormatter can be used for tests started via parallel_tests gem.

Note: Launch id is shared between independent processes (as is the case with parallel_tests gem) via a file in `Dir.tmpdir`.

## Links

- [ReportPortal](https://github.com/reportportal/)
10 changes: 3 additions & 7 deletions lib/report_portal/cucumber/formatter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,10 @@ module Cucumber
class Formatter
# @api private
def initialize(config)
ENV['REPORT_PORTAL_USED'] = 'true'
abotalov marked this conversation as resolved.
Show resolved Hide resolved

@logger = Logger.new(config.out_stream)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logger is not loaded by Ruby itself. So there should be a require statement before it's first used.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

require_relative '../logging/logger'

@logger.level = ReportPortal::Settings.instance.log_level || :warn
setup_message_processing

@io = config.out_stream

[:test_case_started, :test_case_finished, :test_step_started, :test_step_finished, :test_run_finished].each do |event_name|
config.on_event event_name do |event|
process_message(event_name, event)
Expand All @@ -23,8 +21,6 @@ def initialize(config)

def puts(message)
process_message(:puts, message)
@io.puts(message)
@io.flush
end

def embed(*args)
Expand All @@ -34,7 +30,7 @@ def embed(*args)
private

def report
@report ||= ReportPortal::Cucumber::Report.new
@report ||= ReportPortal::Cucumber::Report.new(@logger)
end

def setup_message_processing
Expand Down
14 changes: 0 additions & 14 deletions lib/report_portal/cucumber/parallel_formatter.rb

This file was deleted.

54 changes: 0 additions & 54 deletions lib/report_portal/cucumber/parallel_report.rb

This file was deleted.

Loading