通过测试监听器删除（复制）失败的TestNG结果
与此处发布的解决方案类似， TestNG重试失败的测试不会输出正确的测试结果 ，我试图在onFinish（ITestContext上下文）中使用测试监听器删除（重复）测试结果。
虽然使用context.getFailedTests（）。removeResult（result）删除结果似乎工作正常（结果实际上已被删除），但似乎有“其他一些点”从中拉出结果导致构建仍然失败。
另请注意，当我从上面的文章运行样本测试时（其中有一个重复失败被删除，一个通过测试），我得到的结果是“测试结果”（没有按预期清理）与“套件结果” （重复故障按预期删除）。
而且，报告从哪里获取结果以决定是否使构建失败？ 或者只是它在我清理失败的测试之前拉动结果……？
=============================================== Default test Tests run: 3, Failures: 2, Skips: 0 =============================================== =============================================== Default suite Total tests run: 2, Failures: 1, Skips: 0 =============================================== 编辑：只是为了澄清，我们正在使用maven运行这些测试，他们是IT，所以我们使用故障安全插件运行它们。 问题是即使看起来测试被删除，mvnvalidation仍然无法构建，因为它认为无论如何都会发现构建失败。
而且如果从Eclipse运行这样的测试，即使删除了测试，当套件完成时，仍然会在日志中打印失败。
关于RetryAnalyzer：我根本不会考虑使用RetryAnalyzer良好/最佳实践，但如果您发现自己处于需要解决问题的情况，例如您inheritance了依赖RetryAnalyzer的测试套件，您可能会觉得这很有用。
尝试使用此代码：
ListenerApadter ：
public class MyTestListenerAdapter extends TestListenerAdapter { @Override public void onTestFailure(ITestResult result) { if (result.getMethod().getRetryAnalyzer() != null) { MyRetryAnalyzer retryAnalyzer = (MyRetryAnalyzer)result.getMethod().getRetryAnalyzer(); if(retryAnalyzer.isRetryAvailable()) { result.setStatus(ITestResult.SKIP); } else { result.setStatus(ITestResult.FAILURE); } Reporter.setCurrentTestResult(result); } } @Overrride public void onFinish(ITestContext context) { Iterator failedTestCases =context.getFailedTests().getAllResults().iterator(); while (failedTestCases.hasNext()) { System.out.println("failedTestCases"); ITestResult failedTestCase = failedTestCases.next(); ITestNGMethod method = failedTestCase.getMethod(); if (context.getFailedTests().getResults(method).size() > 1) { System.out.println("failed test case remove as dup:" + failedTestCase.getTestClass().toString()); failedTestCases.remove(); } else { if (context.getPassedTests().getResults(method).size() > 0) { System.out.println("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString()); failedTestCases.remove(); } } } } }
RetryAnalizer ：
public class MyRetryAnalyzer implements IRetryAnalyzer { private static int MAX_RETRY_COUNT = 3; AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT); public boolean isRetryAvailable() { return (count.intValue() > 0); } @Override public boolean retry(ITestResult result) { boolean retry = false; if (isRetryAvailable()) { System.out.println("Going to retry test case: " + result.getMethod() + ", " + (MAX_RETRY_COUNT - count.intValue() + 1) + " out of " + MAX_RETRY_COUNT); retry = true; count.decrementAndGet(); } return retry; } }
POM.xml – > Surefire配置：
这是你应该配置“覆盖”surefire监听器的地方，它有自己的计数器。
org.apache.maven.plugins maven-surefire-plugin 2.18.1 ${basedir}/testng.xml listener Utils.MyTestListenerAdapter,Utils.MyRetryAnalizer
我最终选择了一个使用套件监听器的解决方案。
解决方案可能无法完全清除TestNG /您的测试记录到控制台的内容，但是如果您使用TestNG Jenkins插件，并且每次测试都先失败，然后成功，那么测试运行最终会变为绿色，即I猜是最重要的。
是的，我们运行mvn integration-test （不是mvn verify）并让TestNG插件处理pass / fail。 解决方案非常相似/基于此处发布的内容 。
import java.util.Map; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.testng.ISuite; import org.testng.ISuiteListener; import org.testng.ISuiteResult; import org.testng.ITestContext; /** * {@link ISuiteListener} implementation to clean up duplicate test results caused by retrying tests using the * {@link RetryAnalyzer} */ public class SuiteResultListener implements ISuiteListener { private static final Logger LOG = LogManager.getLogger(); @Override public void onStart(ISuite suite) { } @Override public void onFinish(ISuite suite) { LOG.info("Cleaning up duplicate test failures in suite '" + suite.getName() + "' ..."); final Map results = suite.getResults(); int removedFailures = 0; for (ISuiteResult result : results.values()) { final ITestContext testContext = result.getTestContext(); removedFailures += TestListenerUtil.cleanUpDuplicateFailures(testContext); } LOG.info("Removed " + removedFailures + " duplicate test failure(s) from suite '" + suite.getName() + "'"); } }
这是TestListenerUtil类中发生的魔力：
public static int cleanUpDuplicateFailures(ITestContext testContext) { final String testContextName = testContext.getName(); int removedFailures = 0; LOG.info("Cleaning up failures in test context '" + testContextName + "' ..."); final Set failedTests = testContext.getFailedTests().getAllResults(); if (failedTests.isEmpty()) { LOG.info("There are no failures in test context '" + testContextName + "'\n"); } else { // collect all id's from passed test final Set passedTestIds = new HashSet<>(); final Set passedTests = testContext.getPassedTests().getAllResults(); LOG.info("Analyzing " + passedTests.size() + " passed test(s)"); for (ITestResult result : passedTests) { final int testId = TestListenerUtil.getId(result); passedTestIds.add(testId); LOG.info(" Passed test " + TestListenerUtil.getName(result) + ": #" + testId + " @ " + getStartTime(result)); } // check which failed test results should be removed final List resultsToBeRemoved = new ArrayList<>(); final Set failedTestIds = new HashSet<>(); LOG.info("Analyzing " + failedTests.size() + " failed test(s)"); for (ITestResult result : failedTests) { final int testId = TestListenerUtil.getId(result); final String name = TestListenerUtil.getName(result); // if we saw this test pass or fail before we mark the result for deletion if (failedTestIds.contains(testId) || passedTestIds.contains(testId)) { LOG.info(" Adding test " + name + " to be removed: #" + testId + " @ " + getStartTime(result)); resultsToBeRemoved.add(testId); } else { LOG.info(" Remembering failed test " + name + ": #" + testId + " @ " + getStartTime(result)); failedTestIds.add(testId); } } // finally delete all duplicate failures (if any) final int duplicateFailures = resultsToBeRemoved.size(); if (duplicateFailures > 0) { LOG.info("Cleaning up failed tests (expecting to remove " + resultsToBeRemoved.size() + " result(s)) ..."); for (ITestResult result : testContext.getFailedTests().getAllResults()) { final int testId = TestListenerUtil.getId(result); final String info = TestListenerUtil.getName(result) + ": #" + testId + " @ " + getStartTime(result); if (resultsToBeRemoved.contains(testId)) { LOG.info(" Removing failed test result " + info); testContext.getFailedTests().removeResult(result); resultsToBeRemoved.remove((Integer) testId); removedFailures++; } else { LOG.info(" Not removing failed test result " + info); } } } if (removedFailures == duplicateFailures) { LOG.info("Removed " + removedFailures + " failed test result(s) in '" + testContextName + "'\n"); } else { LOG.warn("Removed " + removedFailures + " failed test result(s) in '" + testContextName + "' (expected to remove " + duplicateFailures + ")\n"); } } return removedFailures; }
使用这两个额外的utils方法：
public static String getName(ITestResult result) { final List parameters = new ArrayList<>(); if (result.getParameters() != null) { for (Object parameter : result.getParameters()) { if (parameter instanceof TestResult && ((TestResult) parameter).getStatus() < 0) { // TestResult.toString() will explode with status < 0, can't use the toString() method parameters.add(parameter.getClass().getName() + "@" + parameter.hashCode()); } else { parameters.add(parameter == null ? "null" : parameter.toString()); } } } return result.getTestClass().getRealClass().getSimpleName() + "." + result.getMethod().getMethodName() + "(" + StringUtils.join(parameters, ",") + ")"; } public static int getId(ITestResult result) { final HashCodeBuilder builder = new HashCodeBuilder(); builder.append(result.getTestClass().getRealClass()); builder.append(result.getMethod().getMethodName()); builder.append(result.getParameters()); return builder.toHashCode(); }
此外，如果您对我们的RetryAnalyzer如何工作感兴趣，请参阅下文。
需要理解的一件事是，我们在RetryAnalyzer和重复结果清理中都考虑了测试方法的参数。 这些是相关的原因我们经常使用DataProviders。
import java.util.HashMap; import java.util.Map; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.testng.IRetryAnalyzer; import org.testng.ITestResult; public class RetryAnalyzer implements IRetryAnalyzer { private static final Logger LOG = LogManager.getLogger(); private static Integer maxRetries; private final Map retryCount = new HashMap<>(); @Override public boolean retry(ITestResult result) { // only re-try failures if (result.getStatus() == ITestResult.FAILURE) { final String testName = TestListenerUtil.getName(result); final int count = getRetryCount(result); final int maxRetriesAllowed = getMaxRetriesAllowed(); if (count < maxRetriesAllowed) { retryCount.put(TestListenerUtil.getId(result), count + 1); LOG.info("Retrying test (attempt " + (count + 1) + "/" + maxRetriesAllowed + "): " + testName); return true; } else { LOG.error("Failing test after " + count + " retries: " + testName); } } return false; } public boolean canRetry(ITestResult result) { return result.getStatus() == ITestResult.FAILURE && getRetryCount(result) < getMaxRetriesAllowed(); } private int getRetryCount(ITestResult result) { final int testId = TestListenerUtil.getId(result); return retryCount.containsKey(testId) ? retryCount.get(testId) : 0; } public static int getMaxRetriesAllowed() { return maxRetries == null ? Config.MAX_TEST_RETRIES : maxRetries; } }