Using Your Cluster

Consult the HPE MLDE docs for full details on using the cluster, and developing models using MLDE.

Authentication

Your cluster can be accessed in one of two ways: through the HPE MLDES web portal, or through the CLI.

In the web portal, click View Cluster to access the cluster's web interface.

To use the CLI, set the DET_MASTER environment variable to the cluster's URL (you can copy it using the clipboard button next to View Cluster) and run det auth login. This will open a page in your browser to authenticate before returning to your terminal.

The CLI can also be used in Jupyter notebooks launched from the web interface. When the session starts, the det CLI will already be configured correctly and authenticated. If your session times out, the notebook will continue running and you will be able to navigate back to it from the Tasks tab in the cluster UI The det CLI will require you to re-authentication by running det auth login --headless and following the instructions.

Access Your Own Datasets

If you have training datasets in S3 or GCS in the same account as your cluster, your HPE MLDES clusters may have access to them already. If they are unable to access them, or if your datasets are stored in a different account, ensure that your buckets are configured with a policy that grants access to the cluster agent role that was provided when setting up the org.

To access other storage, you may want to place the required credentials in the environment variables for your experiment. However, be aware that these will also be accessible to your team.