datasetinsights.commands¶

datasetinsights.commands.download¶

datasetinsights download¶

Download datasets to localhost from known locations. The download command can support downloading from 3 sources usim:// http(s):// gs://

Download from Unity Simulation:

You can specify project_it, run_execution_id, access_token in source-uri

datasetinsights download –source-uri=usim://<access_token>@<project_id>/<run_execution_id> –output=$HOME/data

Alternatively, you can also override access_token such as

datasetinsights download –source-uri=usim://<project_id>/<run_execution_id>: –output=$HOME/data –access-token=<access_token>

Downloading from a http source:

datasetinsights download –source-uri=http://url.to.file.zip: –output=$HOME/data

Downloading from a gcs source:

datasetinsights download –source-uri=gs://url/to/file.zip: –output=$HOME/data

or

datasetinsights download –source-uri=gs://url/to/folder: –output=$HOME/data

datasetinsights download [OPTIONS]

Options

-s, --source-uri <source_uri>¶: Required URI of where this data should be downloaded. Supported source uri patterns ^gs://|^http(s)?://|^usim://

-o, --output <output>¶: Directory on localhost where datasets should be downloaded.

-b, --include-binary¶: Whether to download binary files such as images or LIDAR point clouds. This flag applies to Datasets where metadata (e.g. annotation json, dataset catalog, …) can be separated from binary files.

--access-token <access_token>¶: Unity Simulation access token. This will override synthetic datasets source-uri for Unity Simulation

--checksum-file <checksum_file>¶: Dataset checksum text file path. Path can be a HTTP(S) url or a local file path. This will help check the integrity of the downloaded dataset.

class datasetinsights.commands.download.SourceURI¶

Bases: click.types.ParamType

Source URI Parameter.

Parameters: click ([type]) – [description]
Raises: click.BadParameter – [description]
Returns: [description]
Return type: [type]

PREFIX_PATTERN = '^gs://|^http(s)?://|^usim://'¶

convert(value, param, ctx)¶: Validate source URI and Converts the value.

name = 'source_uri'¶

datasetinsights.commands.evaluate¶

datasetinsights evaluate¶

Start model evaluation tasks for a pre-trained model.

datasetinsights evaluate [OPTIONS]

Options

-c, --config <config>¶: Required Path to the config estimator yaml file.

-p, --checkpoint-file <checkpoint_file>¶: Required URI to a checkpoint file.

-t, --test-data <test_data>¶: Required Directory on localhost where test dataset is located.

-l, --tb-log-dir <tb_log_dir>¶: Path to the directory where tensorboard events should be stored. This Path can be GCS URI (e.g. gs://<bucket>/runs) or full path to a local directory.

-w, --workers <workers>¶: Number of multiprocessing workers for loading datasets. Set this argument to 0 will disable multiprocessing which is recommended when running inside a docker container.

--kfp-metrics-dir <kfp_metrics_dir>¶: Path to the directory where Kubeflow Metrics files are stored.

--kfp-metrics-filename <kfp_metrics_filename>¶: Kubeflow Metrics filename.

--no-cuda¶: Force to disable CUDA. If CUDA is available and this flag is False, model will be trained using CUDA.

datasetinsights.commands.train¶

datasetinsights train¶

Start model training (and optionally validation) tasks.

datasetinsights train [OPTIONS]

Options

-c, --config <config>¶: Required Path to the config estimator yaml file.

-t, --train-data <train_data>¶: Required Directory on localhost where train dataset is located.

-e, --val-data <val_data>¶: Directory on localhost where validation dataset is located.

-p, --checkpoint-file <checkpoint_file>¶: URI to a checkpoint file. If specified, model will load from this checkpoint and resume training.

-l, --tb-log-dir <tb_log_dir>¶: Path to the directory where tensorboard events should be stored. This Path can be GCS URI (e.g. gs://<bucket>/runs) or full path to a local directory.

-p, --checkpoint-dir <checkpoint_dir>¶: Path to the directory where model checkpoint files should be stored. This Path can be GCS URI (e.g. gs://<bucket>/checkpoints) or full path to a local directory.

-w, --workers <workers>¶: Number of multiprocessing workers for loading datasets. Set this argument to 0 will disable multiprocessing which is recommended when running inside a docker container.

--no-cuda¶: Force to disable CUDA. If CUDA is available and this flag is False, model will be trained using CUDA.

--no-val¶: Force to disable validations.

class datasetinsights.commands.Entrypoint(name=None, invoke_without_command=False, no_args_is_help=None, subcommand_metavar=None, chain=False, result_callback=None, **attrs)¶

Bases: click.core.MultiCommand

Click MultiCommand Entrypoint For Datasetinsights CLI

get_command(ctx, name)¶: Dynamically get the command.

list_commands(ctx)¶: Dynamically get the list of commands.