使用集合让一切井井有条
根据您的偏好保存内容并对其进行分类。
Firebase Test Lab for Android 为 American Express 降低了 50% 的应用测试成本
简介
AMEX 移动应用的 Android 版本下载量达到 100 万次,
其信用卡会员使用这款应用来跟踪支出和奖励、查找优惠、
支付账单,并享受应用端专享功能。为了协调频繁的更新,
American Express 建立了一个设备实验室,负责在发布应用代码更改之前,
在所有平台上进行测试。
挑战
由于有数十台设备运行不同的 Android 版本,
该实验室很难扩大规模。考虑到设备实验室的吞吐量有限,
添加新测试会降低工作效率,
而且在有新团队成员加入时,
增加的代码更改量会增加更多的负载。
维护设备实验室既耗时又昂贵,
分散了工程师的注意力,
导致他们无法专注于打造能给持卡会员带来
额外价值的增强功能。这些设备通常存在
需要手动干预的问题,往往会迫使团队重新运行整个测试套件。
他们会定期将有问题的设备
从设备实验室中移除,
不过在等待新设备期间,团队经常被迫中断工作。
解决方案
在产品的 Alpha 版期间,AMEX 应用的更新频率、
对高质量发布版本的承诺以及工程师的协作意愿
给 Firebase Test Lab 团队创造了
理想的环境。而 American Express 提供了宝贵的反馈,
为打造这款产品的发布版本提供了帮助。
Firebase Test Lab 集成了
American Express 含有数百项插桩测试的
复杂套件。早在 Test Lab 处于初期开发阶段时,
American Express 就已经能够将其所有持续集成测试
转移到 Test Lab,成功地
淘汰了其原有设备实验室的绝大部分设备。原本负责维护这些设备的工程师们
转而专注于为信用卡会员开发新功能,
同时编写和运行测试
以确保产品的高品质。
此外,American Express 还能够增加测试数量,
而不必担心拖慢测试套件的执行速度,
也不必费心去为自有设备实验室扩容。这包括
应对新团队成员加入并生成额外的代码更改时,
测试数量的激增。American Express 现在会
针对每项拉取请求运行 1000 多个插桩测试,
相当于每个公开发布版本运行数十万次测试。
运行如此之多的并发测试极大地提高了
他们防范新错误和重复错误的能力,
而这一切都要归功于 Firebase Test Lab 大规模
同时载入测试的能力。
免责声明:实际结果可能会有所不同。
成果
完成的测试量增加 30%,而且用时仅为原本的一半
完整测试套件的执行速度提高 2 倍以上,
添加测试时不会拖慢速度
与内部设备实验室相比,成本降低 50%
借助 Firebase Test Lab,American Express 能够针对每个拉取请求执行我们的整个 Android 测试套件。Firebase 负责调节测试基础架构,使我们的工程师可以专注于代码,全心推动应用的改进和测试套件的相应更改。"
- American Express 副总裁及首席工程师 Michael Bailey
[null,null,[],[],[],null,["[*arrow_back*\nExplore other case studies](/case-studies) \n[View more solutions](/solutions) \n\nFirebase Test Lab for Android reduced American Express app test costs by 50%\n\n*** ** * ** ***\n\nIntroduction The AMEX mobile app, with a million Android downloads, is used by\nCard Members to track spending and rewards, find offers, pay\nbills, and enjoy app-only features. To facilitate frequent\nupdates, American Express built a device lab to test code changes\nin their app across all platforms before release.\n\n*** ** * ** ***\n\nChallenge With dozens of devices running various Android releases, the lab\nwas tough to scale. Adding new tests decreased productivity\nbecause of the device lab's limited throughput, and when new team\nmembers joined, the increased number of code changes added even\nmore load.\n\nMaintaining the device lab was time-consuming and costly,\ndiverting the attention of engineers who otherwise would have\nbeen focused on enhancements to drive additional value for Card\nMembers. The devices commonly had issues that required manual\nintervention, often forcing reruns of the entire test suite.\nProblematic devices were regularly removed from the device lab,\nand replacing them often created down-time while the team awaited\nthe arrival of new devices. \n\n*** ** * ** ***\n\nSolution \nDuring the product's alpha period, the frequency of updates\nto the AMEX app, commitment to high-quality releases, and\nwillingness of engineers to collaborate provided an ideal\nenvironment for the Firebase Test Lab team. American Express,\nin turn, provided valuable feedback that helped to shape the\nreleased version of the product. Firebase Test Lab integrated\nwith American Express' complex suite of hundreds of\ninstrumentation tests. Even while Test Lab was in early\ndevelopment, American Express was able to switch all of their\ncontinuous integration testing to Test Lab and successfully\nretire much of their device lab. This enabled the engineers\nmaintaining the devices to shift their focus to shipping new\nfeatures to Card Members while also writing and running tests\nto ensure high product quality. \nIn addition, American Express was able to scale the number of\ntests without worrying about slowing down test-suite execution\nand scaling up a homegrown device lab. This includes handling\nsurges in the number of tests as new team members join and\ngenerating additional code changes. American Express now runs\nover 1,000 instrumentation tests on every pull request, which\nequals hundreds of thousands of tests per public release.\nRunning so many concurrent tests dramatically increases\ntheir ability to prevent new and recurring bugs, all thanks\nto Firebase Test Lab's ability to massively parallelize\ntesting.\n\nDISCLAIMER: Results may vary.\n\n*** ** * ** ***\n\nResults\n\n30% more tests in half the time\n\n2X+ faster execution of full test suite, with no slowdown when adding\ntests\n\n50% lower cost vs. internal device lab \n\nFirebase Test Lab enables American Express to execute our entire extensive Android test suite for every pull request. Firebase takes care of scaling the test infrastructure, so our engineers can focus on the code, driving improvements to the app and corresponding changes to our test suite.\"\n\n\n- Michael Bailey, VP and Principal Engineer, American Express \nTry Firebase today\n\n\nIntegrating it into your app is easy.\n[Get started](https://console.firebase.google.com/) \n\nAll Firebase products \n\nBuild\n\n- [App Check](/products/app-check)\n- [App Hosting](/products/app-hosting)\n- [Authentication](/products/auth)\n- [Cloud Functions](/products/functions)\n- [Cloud Storage](/products/storage)\n- [Data Connect](/products/data-connect)\n- [Extensions](/products/extensions)\n- [Firestore](/products/firestore)\n- [Firebase ML](/products/ml)\n- [Genkit](https://genkit.dev/)\n- [Hosting](/products/hosting)\n- [Realtime Database](/products/realtime-database)\n- [Firebase AI Logic client SDKs](/products/firebase-ai-logic)\n\n[Generative AI](/products/generative-ai) \n\nRun\n\n- [A/B Testing](/products/ab-testing)\n- [App Distribution](/products/app-distribution)\n- [Cloud Messaging](/products/cloud-messaging)\n- [Crashlytics](/products/crashlytics)\n- [Google Analytics](/products/analytics)\n- [In-App Messaging](/products/in-app-messaging)\n- [Performance Monitoring](/products/performance)\n- [Remote Config](/products/remote-config)\n- [Test Lab](/products/test-lab)"]]