{"templateId":"markdown","sharedDataIds":{"sidebar":"sidebar-@l10n/ja/sidebars.yaml"},"props":{"metadata":{"markdoc":{"tagList":["admonition"]},"type":"markdown"},"seo":{"title":"ML実験トラッキングとモデル管理","description":"Treasure AIのオートML環境における、ML実験のトラッキングとモデル管理の詳細なガイド。実験結果の記録、評価、モデルパフォーマンスの追跡方法を解説。","siteUrl":"https://docs.treasure.ai","lang":"en-US","jsonLd":{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://www.treasure.ai/","name":"Treasure AI","url":"https://www.treasure.ai/","logo":"https://www.treasure.ai/hubfs/assets/images/logos/primary-logo.svg"},{"@type":"WebSite","@id":"https://docs.treasure.ai/#website","name":"Treasure AI Documentation","url":"https://docs.treasure.ai/","inLanguage":["en","ja"],"publisher":{"@id":"https://www.treasure.ai/"}}]}},"dynamicMarkdocComponents":[],"compilationErrors":[],"ast":{"$$mdtype":"Tag","name":"article","attributes":{},"children":[{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"ml実験トラッキングとモデル管理","__idx":0},"children":["ML実験トラッキングとモデル管理"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["ML実験トラッキングは、機械学習実験の結果を整理、記録、分析するプロセスです。このドキュメントでは、ML実験トラッキングを可能にするワークフローの作成方法について説明します。"]},{"$$mdtype":"Tag","name":"Admonition","attributes":{"type":"info"},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":["完全なML実験トラッキングワークフローのコードは、",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://github.com/treasure-data/treasure-boxes/blob/automl/machine-learning-box/automl/ml_experiment.dig"},"children":["Treasure Boxes"]},"で確認できます。"]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["目次"]}]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"#track-ml-experiments"},"children":["ML実験のトラッキング"]}]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"#record-evaluation-results-for-each-model"},"children":["各モデルの評価結果の記録"]}]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"#detect-drift-in-model-performance-over-time"},"children":["時系列でのモデルパフォーマンスのドリフト検出"]}]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"ml実験のトラッキング","__idx":1},"children":["ML実験のトラッキング"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["ベストプラクティスとして、エンドツーエンドのデータ処理ワークフローの一部として、トレーニングタスクに続いて「",{"$$mdtype":"Tag","name":"em","attributes":{},"children":["track_experiment"]},"」タスクを使用して、各ML実験をトラッキングする必要があります。_track_experiment_タスクは、ML実験情報とモデル名を「automl_experiments」という名前のTDテーブルに記録するSQLクエリを発行します。サンプルワークフローコードは以下の通りです："]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"yaml","header":{"controls":{"copy":{}}},"source":"+create_db_tbl_if_not_exists:\n  td_ddl>: null\n  create_databases:\n    - '${ output_database}'\n  create_tables:\n    - automl_experiments\n    - automl_eval_results\n+train:\n  ml_train>:\n    docker:\n      task_mem: 128g\n    notebook: gluon_train\n    model_name: 'gluon_model_${session_id}'\n    input_table: '${input_database}.${train_data_table}'\n    target_column: '${target_column}'\n    time_limit: '${fit_time_limit}'\n    share_model: true\n    export_leaderboard: '${output_database}.leaderboard_${train_data_table}'\n    export_feature_importance: '${output_database}.feature_importance_${train_data_table}'\n+track_experiment:\n  td>: queries/track_experiment.sql\n  insert_into: '${output_database}.automl_experiments'\n  last_executed_notebook: '${automl.last_executed_notebook}'\n  user_id: '${automl.last_executed_user_id}'\n  user_email: '${automl.last_executed_user_email}'\n  model_name: 'gluon_model_${session_id}'\n  shared_model: '${automl.shared_model}'\n  task_attempt_id: '${attempt_id}'\n  session_time: '${session_local_time}'\n  engine: presto\n","lang":"yaml"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["上記のワークフローコードは、_automl_experiments_テーブルに以下のようなコンテンツを生成します："]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"task_attempt_id"},"children":["task_attempt_id"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"session_time"},"children":["session_time"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"user_id"},"children":["user_id"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"user_email"},"children":["user_email"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"model_name"},"children":["model_name"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"shared_model"},"children":["shared_model"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"notebook_url"},"children":["notebook_url"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["849779333"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-05-18 7:19:18"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["7776"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["xxx@treasure-data.com"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_161722236"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["b4a568da-e6f3-4057-b694-e2e19bf0e924"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["https://console.treasuredata.com/app/workflows/automl/notebook/4a3c431b3aea4705b32a47d85ca46368"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["849772621"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-05-18 7:08:30"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["7776"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["xxx@treasure-data.com"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_161721046"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["94ad5d0e-89ac-4836-99c4-2bc8f975ccbe"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["https://console.treasuredata.com/app/workflows/automl/notebook/b390b932d4a64fd3a2dc3b75503430fb"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["849768123"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-05-18 7:01:13"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["7777"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["yyy@treasure-data.com"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_161720337"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["4f2351a3-dd8c-418e-8057-4c8ec9a90cbe"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["https://console.treasuredata.com/app/workflows/automl/notebook/e8b3319c982345a48ff74db0003d7c9c"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["849760942"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-05-18 6:49:50"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["7776"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["xxx@treasure-data.com"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_161718676"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["93e68b09-1a2f-4049-bb89-2bfe596ca9b3"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["https://console.treasuredata.com/app/workflows/automl/notebook/b02959b1469e4b9c86ec6c6809acc5ff"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["849753199"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-05-18 6:36:36"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["7776"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["xxx@treasure-data.com"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_161717236"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["a7e456d3-8fcf-4173-afb7-f2d58bb985cd"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["https://console.treasuredata.com/app/workflows/automl/notebook/d3dcbbab99774bd594106a496ec2b2ab"]}]}]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["テーブルの各レコードには、モデル名、モデルを作成したユーザーの詳細、モデルが作成されたセッション時刻、および生成されたノートブックへのリンクが含まれます。"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"各モデルの評価結果の記録","__idx":2},"children":["各モデルの評価結果の記録"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["オプションで、評価データセットを使用して各モデルの品質を記録できます。次のワークフローは、分類問題の標準的な評価指標である",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://en.wikipedia.org/wiki/Receiver_operating_characteristic"},"children":["AUROC"]},"を使用してモデル品質を記録する例です。",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["record_evaluation"]},"タスクは、automl_eval_resultsテーブルに評価結果を記録します。"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"yaml","header":{"controls":{"copy":{}}},"source":"+predict:\n  ml_predict>:\n    docker:\n      task_mem: 64g\n    notebook: gluon_predict\n    model_name: 'gluon_model_${session_id}'\n    input_table: '${input_database}.${test_data_table}'\n    output_table: '${output_database}.predicted_${test_data_table}_${session_id}'\n+evaluation:\n  td>: queries/auc.sql\n  table: '${output_database}.predicted_${test_data_table}_${session_id}'\n  target_column: '${target_column}'\n  positive_class: ' >50K'\n  store_last_results: true\n  engine: hive\n+record_evaluation:\n  td>: queries/record_evaluation.sql\n  insert_into: '${output_database}.automl_eval_results'\n  engine: presto\n  model_name: 'gluon_model_${session_id}'\n  test_table: '${input_database}.${test_data_table}'\n  session_time: '${session_local_time}'\n  auc: '${td.last_results.auc}'\n","lang":"yaml"},"children":[]},{"$$mdtype":"Tag","name":"Admonition","attributes":{"type":"info"},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Treasure DataのHive実行エンジンは、多数の評価指標をサポートするHivemallをサポートしています。詳細については、",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://hivemall.github.io/eval/binary_classification_measures.html"},"children":["Hivemallドキュメント"]},"を参照してください。"]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["「automl_eval_results」テーブルのコンテンツ例："]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"session_time"},"children":["session_time"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"model_name"},"children":["model_name"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"ml_datasets.gluon_test"},"children":["ml_datasets.gluon_test"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"auroc"},"children":["auroc"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-06-06 6:21:40"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_164947310"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["ml_datasets.gluon_test"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["0.9226243033"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-06-14 6:49:22"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_166350110"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["ml_datasets.gluon_test"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["0.9299335758"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-06-15 7:35:30"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_166532223"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["ml_datasets.gluon_test"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["0.9300292252"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["2023-05-18 7:19:18"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["gluon_model_161722236"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["ml_datasets.gluon_test"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["0.9238149699"]}]}]}]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"時系列でのモデルパフォーマンスのドリフト検出","__idx":3},"children":["時系列でのモデルパフォーマンスのドリフト検出"]},{"$$mdtype":"Tag","name":"Admonition","attributes":{"type":"info"},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":["「ドリフト」は、機械学習において、機械学習モデルのパフォーマンスが時間の経過とともにゆっくりと悪化または陳腐化することを表す用語です。ドリフトには主に2つのタイプがあります：データドリフトと",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://en.wikipedia.org/wiki/Concept_drift"},"children":["コンセプトドリフト"]},"です。データドリフトとコンセプトドリフトの両方が、機械学習モデルのパフォーマンス低下につながる可能性があります。"]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["次のワークフロータスクを使用することで、各モデルの精度と品質を記録し、データとモデルパフォーマンスのドリフトを検出できます。スケジュールされたワークフロージョブを使用して、モデルパフォーマンスを追跡し、モデルパフォーマンスがドリフトした場合に警告を発することができます。"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["ドリフト検出にはいくつかのスキームがあります。次のワークフロー例は、評価指標を使用してMLモデルパフォーマンスの劣化を特定する方法を示しています。ドリフトが検出されると、次のようにアラートメールをトリガーできます："]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"yaml","header":{"controls":{"copy":{}}},"source":"# timezone: PST\n# schedule:\n#  daily>: 07:00:00\n+evaluation:\n  td>: queries/auc.sql\n  table: '${output_database}.predicted_${test_data_table}_${session_id}'\n  target_column: '${target_column}'\n  positive_class: ' >50K'\n  store_last_results: true\n  engine: hive\n+alert_if_drift_detected:\n  if>: '${td.last_results.auc < 0.93}'\n  _do: null\nmail>: null\ndata: 'Detect drift in model performance. AUC was ${td.last_results.auc}.'\nsubject: Drift detected\nto:\n  - me@example.com\nbcc:\n  - foo@example.com\n  - bar@example.com\n","lang":"yaml"},"children":[]},{"$$mdtype":"Tag","name":"Admonition","attributes":{"type":"info"},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":["ドリフト検出のために",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://docs.digdag.io/scheduling_workflow.html?highlight=schedule"},"children":["ワークフロー実行をスケジュール"]},"できます。また、ドリフトが検出された場合、",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://docs.digdag.io/operators/if.html"},"children":["条件演算子"]},"を使用してアラートメールを送信したり、モデルを再構築したりできます。"]}]}]},"headings":[{"value":"ML実験トラッキングとモデル管理","id":"ml実験トラッキングとモデル管理","depth":1},{"value":"ML実験のトラッキング","id":"ml実験のトラッキング","depth":1},{"value":"各モデルの評価結果の記録","id":"各モデルの評価結果の記録","depth":1},{"value":"時系列でのモデルパフォーマンスのドリフト検出","id":"時系列でのモデルパフォーマンスのドリフト検出","depth":1}],"frontmatter":{"seo":{"title":"ML実験トラッキングとモデル管理","description":"Treasure AIのオートML環境における、ML実験のトラッキングとモデル管理の詳細なガイド。実験結果の記録、評価、モデルパフォーマンスの追跡方法を解説。"}},"lastModified":"2026-06-17T07:22:53.000Z","pagePropGetterError":{"message":"","name":""}},"slug":"/ja/products/customer-data-platform/machine-learning/automl/advanced-topics/ml-experiment-tracking-and-model-management","userData":{"isAuthenticated":false,"teams":["anonymous"]},"isPublic":true}