You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The update of the test projects (see pmd/pmd#3640) is applied already, since this integration test just runs on pmd from master branch. We probably need to update our patch files, which simulate some changes.
In detail:
test_case_1_single_java_rule_changed
2021-11-29T10:12:00.4054476Z {:errors=>{:changed=>0, :new=>0, :removed=>1, :base_total=>40, :patch_total=>39}, :violations=>{:changed=>0, :new=>0, :removed=>512, :base_total=>512, :patch_total=>0}, :configerrors=>{:changed=>0, :new=>0, :removed=>3, :base_total=>3, :patch_total=>0}}
2021-11-29T10:12:00.4055486Z #############################
2021-11-29T10:12:00.4056013Z F
2021-11-29T10:12:00.4056599Z ===============================================================================
2021-11-29T10:12:00.4057716Z Failure: test_case_1_single_java_rule_changed(ManualIntegrationTests)
2021-11-29T10:12:00.4059145Z test/manual_integration_tests.rb:48:in `test_case_1_single_java_rule_changed'
2021-11-29T10:12:00.4060281Z 45: assert_equal(0, @summary[:violations][:changed], 'found changed violations')
2021-11-29T10:12:00.4061383Z 46: assert_equal(0, @summary[:violations][:new], 'found new violations')
2021-11-29T10:12:00.4063044Z 47: # These are the artificially created false-negatives for AbstractClassWithoutAbstractMethod rule
2021-11-29T10:12:00.4064646Z => 48: assert_equal(34 + 234, @summary[:violations][:removed], 'found removed violations')
2021-11-29T10:12:00.4065449Z 49:
2021-11-29T10:12:00.4066137Z 50: # errors might have been caused in the baseline for other rules (only visible in the stacktrace)
2021-11-29T10:12:00.4066976Z 51: # hence they might appear as removed
2021-11-29T10:12:00.4067543Z found removed violations
2021-11-29T10:12:00.4068034Z <268> expected but was
2021-11-29T10:12:00.4068433Z <512>
We have now many more removed violations (512 instead of 268). This could be just because of the changed projects - the baseline probably has now more violations that are affected...
test case 2 - single xpath rule changed
2021-11-29T10:21:56.1686223Z �[0;0m#############################: test_case_2_single_xpath_rule_changed
2021-11-29T10:21:56.1687267Z {:errors=>{:changed=>0, :new=>0, :removed=>1, :base_total=>40, :patch_total=>39}, :violations=>{:changed=>0, :new=>15, :removed=>477, :base_total=>64130, :patch_total=>63668}, :configerrors=>{:changed=>0, :new=>0, :removed=>3, :base_total=>3, :patch_total=>0}}
2021-11-29T10:21:56.1688397Z #############################
2021-11-29T10:21:56.1688730Z F
2021-11-29T10:21:56.1689251Z ===============================================================================
2021-11-29T10:21:56.1689942Z Failure: test_case_2_single_xpath_rule_changed(ManualIntegrationTests)
2021-11-29T10:21:56.1691218Z test/manual_integration_tests.rb:81:in `test_case_2_single_xpath_rule_changed'
2021-11-29T10:21:56.1691974Z 78: print "#############################: test_case_2_single_xpath_rule_changed\n" \
2021-11-29T10:21:56.1692664Z 79: "#{@summary}\n#############################\n"
2021-11-29T10:21:56.1693470Z 80: assert_equal(0, @summary[:violations][:changed], 'found changed violations')
2021-11-29T10:21:56.1694413Z => 81: assert_equal(0, @summary[:violations][:new], 'found new violations')
2021-11-29T10:21:56.1695266Z 82: # There are no violations, that have been removed for AvoidMessageDigestField
2021-11-29T10:21:56.1696337Z 83: assert_equal(0, @summary[:violations][:removed], 'found removed violations')
2021-11-29T10:21:56.1696872Z 84:
2021-11-29T10:21:56.1697229Z found new violations
2021-11-29T10:21:56.1697645Z <0> expected but was
2021-11-29T10:21:56.1697997Z <15>
That's weird - by disabled one xpath rule we got new violations - that shouldn't happen.
Update integration tests #97
* pr-97:
Update integration tests after pmd/pmd#3666 has been merged
[ci] Use new cache
Update release notes (#96)
Update integration tests
[ci] Run these integration tests monthly automatically
Update test case 5
Use en_US.UTF-8 as locale, don't sign the commits
Fix test case 2
Fix test case 1
Integration tests need to be updates, they are failing right now: https://github.com/pmd/pmd-regression-tester/actions/workflows/manual-integration-tests.yml
The update of the test projects (see pmd/pmd#3640) is applied already, since this integration test just runs on pmd from master branch. We probably need to update our patch files, which simulate some changes.
In detail:
We have now many more removed violations (512 instead of 268). This could be just because of the changed projects - the baseline probably has now more violations that are affected...
That's weird - by disabled one xpath rule we got new violations - that shouldn't happen.
Here we got again new violations, while we expected no change at all.
That's fine, this test case still works.
The text was updated successfully, but these errors were encountered: