Test::Unit Output is kind of... ugh.

Posted by Shane Becker on 2008-01-24 at 4:19am

<h4><strong>WARNING! WARNING! WARNING!</strong></h4>
<h4>Serious computer geekery approaching. Non-geeks, <a href="http://icanhascheezburger.com/" title="Lolcats ‘n’ Funny Pictures - I Can Has Cheezburger?">abort now!</a>

</h4>

<p>
  The <a href="http://rubyonrails.com/" title="Ruby on Rails">Ruby on Rails</a> app at the place that I work has a whole fuck ton of models, like over 150. So when we run our tests there's a lines and lines of output just telling us which files are being hit. You know what? I don't care. Just show me dots. DOTS!
</p>

<p>Our output looks like this for all tests passing:</p>
<code>
  <pre> (in /Users/sb/work/vendo/branches/trunk) /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:test "/Library/Ruby/Gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader.rb" "test/unit/account_test.rb" "test/unit/account_theme_composite_test.rb" "test/unit/action_mailer_to_file_test.rb" "test/unit/ad_helper_test.rb" "test/unit/ad_words_campaign_factory_test.rb" "test/unit/ad_words_campaign_test.rb" "test/unit/ad_words_publisher_test.rb" "test/unit/address_test.rb" "test/unit/advertising_dot_com_publisher_test.rb" "test/unit/advertising_dot_com_result_test.rb" "test/unit/all_recipes_campaign_factory_test.rb" "test/unit/all_recipes_campaign_test.rb" "test/unit/audience_test.rb" "test/unit/auth_cookie_test.rb" "test/unit/baseline_estimator_test.rb" "test/unit/billing_transaction_test.rb" "test/unit/binomial_test.rb" "test/unit/blue_lithium_parser_test.rb" "test/unit/bookmarked_creative_theme_test.rb" "test/unit/campaign_creative_click_site_test.rb" "test/unit/campaign_creative_event_test.rb" "test/unit/campaign_creative_result_test.rb" "test/unit/campaign_creative_test.rb" "test/unit/campaign_group_factory_test.rb" "test/unit/campaign_group_test.rb" "test/unit/campaign_mailer_test.rb" "test/unit/campaign_request_mailer_test.rb" "test/unit/campaign_test.rb" "test/unit/campaigns_logs_test.rb" "test/unit/cart_test.rb" "test/unit/cent_accessor_test.rb" "test/unit/color_test.rb" "test/unit/contact_test.rb" "test/unit/country_test.rb" "test/unit/creative_file_test.rb" "test/unit/creative_test.rb" "test/unit/creative_theme_test.rb" "test/unit/credit_card_test.rb" "test/unit/customer_service_mailer_test.rb" "test/unit/customizer/creative_theme_test.rb" "test/unit/customizer_test.rb" "test/unit/dart_parser_test.rb" "test/unit/database_function_test.rb" "test/unit/difference_test.rb" "test/unit/dimension_test.rb" "test/unit/estimator_base_test.rb" "test/unit/event_router_test.rb" "test/unit/extensions/active_record_base_test.rb" "test/unit/extensions/array_test.rb" "test/unit/extensions/date_test.rb" "test/unit/extensions/module_test.rb" "test/unit/extensions/object_test.rb" "test/unit/extensions/time_test.rb" "test/unit/favorite_test.rb" "test/unit/file_helper_test.rb" "test/unit/formatter_test.rb" "test/unit/generic_parser_test.rb" "test/unit/google_result_test.rb" "test/unit/google_sync_mailer_test.rb" "test/unit/hash_test.rb" "test/unit/hush_test.rb" "test/unit/industry_test.rb" "test/unit/keyword_test.rb" "test/unit/language_test.rb" "test/unit/location_node_test.rb" "test/unit/location_test.rb" "test/unit/mime_magick_test.rb" "test/unit/must_test.rb" "test/unit/navigation_test.rb" "test/unit/notification_test.rb" "test/unit/parse_log_test.rb" "test/unit/permission_test.rb" "test/unit/plan_test.rb" "test/unit/publisher_format_test.rb" "test/unit/publisher_test.rb" "test/unit/publisher_theme_errors_test.rb" "test/unit/question_test.rb" "test/unit/region_test.rb" "test/unit/right_media_campaign_factory_test.rb" "test/unit/right_media_campaign_synchronizer_test.rb" "test/unit/role_test.rb" "test/unit/serialization_action_test.rb" "test/unit/simple_account_test.rb" "test/unit/simple_campaign_test.rb" "test/unit/simple_node_test.rb" "test/unit/skin_test.rb" "test/unit/socket_site_parser_test.rb" "test/unit/tag_test.rb" "test/unit/theme_mailer_test.rb" "test/unit/theme_manager_test.rb" "test/unit/time_machine_test.rb" "test/unit/traction_impression_test.rb" "test/unit/traction_test.rb" "test/unit/user_notifier_test.rb" "test/unit/user_test.rb" "test/unit/zillow_result_test.rb" loading... Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader Started .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. Finished in 56.895401 seconds.

546 tests, 2011 assertions, 0 failures, 0 errors /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:test “/Library/Ruby/Gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader.rb” “test/functional/accounts_controller_test.rb” “test/functional/admin_controller_test.rb” “test/functional/ads_controller_test.rb” “test/functional/all_recipes_campaign_groups_controller_test.rb” “test/functional/application_controller_test.rb” “test/functional/bookmarks_controller_test.rb” “test/functional/buttons_controller_test.rb” “test/functional/campaign_creative_result_controller_test.rb” “test/functional/campaign_event_controller_test.rb” “test/functional/campaign_group_controller_test.rb” “test/functional/cart_controller_test.rb” “test/functional/clash_controller_test.rb” “test/functional/creative_logos_controller_test.rb” “test/functional/credit_cards_controller_test.rb” “test/functional/dimensions_controller_test.rb” “test/functional/favorites_controller_test.rb” “test/functional/help_controller_test.rb” “test/functional/helpers/shared/require_permissions_helper_test.rb” “test/functional/helpers/shared/skin_helper_test.rb” “test/functional/helpers/shared/table_helper_test.rb” “test/functional/layout_test.rb” “test/functional/layouts/ad_ready_footer_layout_test.rb” “test/functional/layouts/ad_ready_header_layout_test.rb” “test/functional/layouts/ad_ready_layout_test.rb” “test/functional/meta_controller_test.rb” “test/functional/metrics_controller_test.rb” “test/functional/password_token_controller_test.rb” “test/functional/publishers_controller_test.rb” “test/functional/reports/account_summary_controller_test.rb” “test/functional/reports/ad_summary_controller_test.rb” “test/functional/reports/billing_transaction_controller_test.rb” “test/functional/reports/budget_controller_test.rb” “test/functional/reports/campaign_publisher_summary_controller_test.rb” “test/functional/reports/campaign_summary_controller_test.rb” “test/functional/reports/click_sites_controller_test.rb” “test/functional/reports/creative_attributes_controller_test.rb” “test/functional/reports/creative_theme_search_controller_test.rb” “test/functional/reports/dashboard_controller_test.rb” “test/functional/reports/data_dump_controller_test.rb” “test/functional/reports/inactive_campaign_controller_test.rb” “test/functional/roles_controller_test.rb” “test/functional/sessions_controller_test.rb” “test/functional/signups_controller_test.rb” “test/functional/theme_controller_test.rb” “test/functional/tips_controller_test.rb” “test/functional/tractions_controller_test.rb” “test/functional/users_controller_test.rb” Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader Started ………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………… Finished in 55.987594 seconds.

327 tests, 1058 assertions, 0 failures, 0 errors /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:test “/Library/Ruby/Gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader.rb” “test/integration/admin_test.rb” “test/integration/billing_integration_test.rb” “test/integration/billing_simulator_test.rb” “test/integration/existing_user_create_campaign_test.rb” “test/integration/landing_page_test.rb” “test/integration/new_user_create_all_recipes_campaign_test.rb” “test/integration/new_user_create_campaign_test.rb” “test/integration/rightmedia_test.rb” “test/integration/salesforce_test.rb” “test/integration/save_for_later_process_test.rb” “test/integration/traction_test.rb” “test/integration/update_campaign_test.rb” Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader Started ……………………… Finished in 51.627323 seconds.

27 tests, 283 assertions, 0 failures, 0 errors </pre> </code>

<p>With line wrapping on in the terminal, that's a lot that is getting pushed out of the window.</p>

<p>I would like the output to look like this:</p>

<code>
  <pre> Started : Unit .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. 546 tests, 2011 assertions, 0 failures, 0 errors

Started : Functional ………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………… 327 tests, 1058 assertions, 0 failures, 0 errors

Started : Integration ……………………… 27 tests, 283 assertions, 0 failures, 0 errors </pre> </code>

<p>
  I don't need this to be the default output. I'm fine with using a -q / --quiet flag. I understand a lot of that output is around for debugging kind of stuff, but most of the time you don't need to debug that your tests are being ran properly, but rather that they pass/fail/error. When a fail/error occurs, show me the failing/erring test and stack trace. That's fine. Although, I think that could be cleaned up, too. You know, like --quiet-stack-trace or something. I'll post later how I think stack traces could be leaner in test output. Though, I guess <a href="http://rubini.us/" title="A Ruby virtual machine with teeth">Rubinius</a> is already doing <a href="http://flickr.com/photos/veganstraightedge/2215263727/" title="rubinius back trace slide on Flickr - Photo Sharing!">some bad ass back traces</a> now, but it's not exactly the same thing.
</p>

<p>My home boy, <a href="http://tenderlovemaking.com/">Aaron</a>, satisfied the bulk of our frustration the other night at <a href="http://seattlerb.org/">nerd party</a> by submitting a small patch to <a href="http://zentest.rubyforge.org/ZenTest/classes/Autotest.html">autotest</a> that take a -q flag and doesn't spit out all the test file paths. That was the <a href="http://en.wikipedia.org/wiki/Pareto_principle">80/20</a> win. So that's pretty dope. Now we just need to sort out some dependency issues so we can actually use <a href="http://nubyonrails.com/articles/autotest-rails">autotest</a>.</p>

Dolphins being awesome.

Test::Unit Output is kind of... ugh.

A Pretty Good Bus Ride

Samuel Maverick

On old projects and revisiting them...