hopsworks.environment #

Environment #

Returned by

description `property` #

Description of the environment.

name `property` #

Name of the environment.

python_version `property` #

Python version of the environment.

delete #

delete()

Delete the environment.

Potentially dangerous operation

This operation deletes the python environment.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

inspect_npm_git #

inspect_npm_git(
    url: str,
    ref: str | None = None,
    include_dev_dependencies: bool = False,
    path: str | None = None,
) -> dict

Read and resolve the package.json files of a git repository, without installing.

The repository is cloned, without checkout, by a short-lived job in the project with the git credentials configured under your account settings, so a private repository works once its provider is set up there. Every package.json in the tree is returned, one for a plain repository and one per workspace for a monorepo, resolved against the nearest lockfile.

info = env.inspect_npm_git("https://github.com/acme/mytool", ref="main")
print(info["commit"], info["rootPackageName"])
for manifest in info["manifests"]:
    print(manifest["path"], [p["name"] for p in manifest["resolved"]["packages"]])

PARAMETER	DESCRIPTION
`url`	The https clone URL. TYPE: `str`
`ref`	A branch, tag or commit; the repository's default branch when unset. TYPE: `str \| None` DEFAULT: `None`
`include_dev_dependencies`	Whether `devDependencies` are part of the result. TYPE: `bool` DEFAULT: `False`
`path`	The package.json to read, relative to the repository root, for example `apps/web/package.json` or its directory `apps/web`. Unset reads every package.json in the repository. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`dict`	A dict with `url`, `commit`, `branch`, `rootPackageName`, `rootPackageVersion`,
`dict`	`packageManager`, `warnings` and `manifests`, each manifest carrying its `path` and
`dict`	a `resolved` result in the shape `resolve_npm_package_json` returns.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

install_npm #

install_npm(
    packages: str | list[str | tuple[str, str] | dict],
    flags: list[str] | None = None,
    await_installation: bool | None = True,
    timeout: float | None = None,
) -> library.Library

Install npm packages into the environment as one operation and one image build.

The packages are installed globally with npm, so they are on PATH and importable from any Node.js app, job or deployment the environment runs. The whole list is one operation and one build: resolving and installing fifty packages takes one image layer, not fifty. While it runs, a request of several packages is listed under one placeholder entry. Once the build has finished, the installed packages replace it in the environment's listing.

import hopsworks

project = hopsworks.login()
env_api = project.get_environment_api()
env = env_api.get_environment("my_custom_environment")

env.install_npm(["express@4.19.2", "lodash@4.17.21", "@types/node@20.11.0"])

# Pin a package to a git commit, by the name its package.json declares.
env.install_npm([{"name": "mytool",
                  "git_url": "https://github.com/acme/mytool",
                  "git_ref": "0123456789abcdef0123456789abcdef01234567"}])

PARAMETER	DESCRIPTION
`packages`	What to install. Each entry is `"name@version"` (an exact version or a dist-tag such as `latest`; a bare `"name"` installs `latest`), a `(name, version)` tuple, or a dict with `name` and `version`, or with `name`, `git_url` and `git_ref` for a repository installed as a package, pinned to a full commit. TYPE: `str \| list[str \| tuple[str, str] \| dict]`
`flags`	npm flags for the install, from the supported set: `--ignore-scripts`, `--legacy-peer-deps`, `--no-audit`, `--no-fund`, `--no-optional`, `--strict-peer-deps`. TYPE: `list[str] \| None` DEFAULT: `None`
`await_installation`	If `True` the method returns only when the installation finishes. TYPE: `bool \| None` DEFAULT: `True`
`timeout`	Seconds to wait for the installation to finish before raising. Falls back to the engine's default when unset. It also bounds the wait for any environment command already in flight, which happens before the install is submitted and therefore applies even when `await_installation` is `False`. TYPE: `float \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`library.Library`	The library the operation is tracked under: the package itself for a request of one,
`library.Library`	a placeholder entry for a request of several.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`ValueError`	If an entry cannot be read as a package.

install_npm_from_git #

install_npm_from_git(
    url: str,
    ref: str | None = None,
    manifests: list[str] | None = None,
    install_repository: bool = False,
    include_dev_dependencies: bool = False,
    flags: list[str] | None = None,
    await_installation: bool | None = True,
    timeout: float | None = None,
    *,
    path: str | None = None,
    skip_unresolved: bool = False,
    include_optional_dependencies: bool = False,
) -> library.Library

Install what a git repository's package.json files depend on, and optionally the repository itself, as one build.

# The dependencies of the repository's package.json, at the branch's current commit.
env.install_npm_from_git("https://github.com/acme/mytool", ref="main")

# One workspace of a monorepo, plus the repository installed as a package.
env.install_npm_from_git("https://github.com/acme/mono", manifests=["packages/cli/package.json"],
                         install_repository=True)

PARAMETER	DESCRIPTION
`url`	The https clone URL. TYPE: `str`
`ref`	A branch, tag or commit; the repository's default branch when unset. The install is pinned to the commit the ref points at when this is called. TYPE: `str \| None` DEFAULT: `None`
`manifests`	The package.json paths to install the dependencies of, as `inspect_npm_git` lists them. All of them when unset; an empty list installs none, which with `install_repository` installs only the repository. TYPE: `list[str] \| None` DEFAULT: `None`
`install_repository`	Also install the repository itself as a package, by the name its root package.json declares. The install runs npm; a repository that needs yarn, pnpm or Bun to build is reported but still attempted. TYPE: `bool` DEFAULT: `False`
`include_dev_dependencies`	Also install `devDependencies`. TYPE: `bool` DEFAULT: `False`
`flags`	npm flags for the install; see `install_npm`. TYPE: `list[str] \| None` DEFAULT: `None`
`await_installation`	If `True` the method returns only when the installation finishes. TYPE: `bool \| None` DEFAULT: `True`
`timeout`	Seconds to wait for the installation to finish before raising; see `install_npm`. TYPE: `float \| None` DEFAULT: `None`
`path`	The one package.json to read, relative to the repository root; see `inspect_npm_git`. TYPE: `str \| None` DEFAULT: `None`
`skip_unresolved`	Install what resolved and leave out what did not, instead of raising. TYPE: `bool` DEFAULT: `False`
`include_optional_dependencies`	Also install the `optionalDependencies`; see `install_npm_from_package_json`. The shorter form of `manifests=[path]` that also skips reading the rest of a monorepo. TYPE: `bool` DEFAULT: `False`

RETURNS	DESCRIPTION
`library.Library`	The library the operation is tracked under; see `install_npm`.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`ValueError`	If a manifest is not in the repository, a dependency could not be resolved and `skip_unresolved` is not set, the repository has no name to install it under, or nothing is left to install.

install_npm_from_package_json #

install_npm_from_package_json(
    path: str | None = None,
    lockfile_path: str | None = None,
    include_dev_dependencies: bool = False,
    flags: list[str] | None = None,
    await_installation: bool | None = True,
    timeout: float | None = None,
    *,
    content: str | None = None,
    lockfile_content: str | None = None,
    lockfile_name: str | None = None,
    skip_unresolved: bool = False,
    include_optional_dependencies: bool = False,
) -> library.Library

Install the dependencies a package.json lists, resolved to exact versions, as one build.

See resolve_npm_package_json for how versions are chosen. Dependencies the resolver skips (workspace siblings, aliases, git, file and URL specs) are left out. A dependency it cannot resolve (the registry has no matching version, or could not be reached) fails the call unless skip_unresolved is set. optionalDependencies are left out unless asked for: npm skips an optional dependency it cannot install, but an explicit install of one fails, and the common case is a platform-specific package such as fsevents.

env.install_npm_from_package_json("package.json", lockfile_path="package-lock.json")

PARAMETER	DESCRIPTION
`path`	The package.json: a local path, or a path in the project when no such local file exists. TYPE: `str \| None` DEFAULT: `None`
`lockfile_path`	The lockfile next to it, local or in the project. TYPE: `str \| None` DEFAULT: `None`
`include_dev_dependencies`	Also install `devDependencies`. TYPE: `bool` DEFAULT: `False`
`flags`	npm flags for the install; see `install_npm`. TYPE: `list[str] \| None` DEFAULT: `None`
`await_installation`	If `True` the method returns only when the installation finishes. TYPE: `bool \| None` DEFAULT: `True`
`timeout`	Seconds to wait for the installation to finish before raising; see `install_npm`. TYPE: `float \| None` DEFAULT: `None`
`content`	The package.json content, instead of `path`. TYPE: `str \| None` DEFAULT: `None`
`lockfile_content`	The lockfile content, instead of `lockfile_path`; needs `lockfile_name`. TYPE: `str \| None` DEFAULT: `None`
`lockfile_name`	The lockfile's file name when passed as content. TYPE: `str \| None` DEFAULT: `None`
`skip_unresolved`	Install what resolved and leave out what did not, instead of raising. TYPE: `bool` DEFAULT: `False`
`include_optional_dependencies`	Also install the `optionalDependencies`. TYPE: `bool` DEFAULT: `False`

RETURNS	DESCRIPTION
`library.Library`	The library the operation is tracked under; see `install_npm`.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`ValueError`	If a dependency could not be resolved and `skip_unresolved` is not set, or nothing is left to install.

install_requirements #

install_requirements(
    path: str,
    await_installation: bool | None = True,
    timeout: float | None = None,
)

Install libraries specified in a requirements.txt file.

import hopsworks

project = hopsworks.login()

# Upload to Hopsworks
ds_api = project.get_dataset_api()
requirements_path = ds_api.upload("requirements.txt", "Resources")

# Install
env_api = project.get_environment_api()
env = env_api.get_environment("my_custom_environment")

env.install_requirements(requirements_path)

PARAMETER	DESCRIPTION
`path`	The path in Hopsworks where the `requirements.txt` file is located. TYPE: `str`
`await_installation`	If `True` the method returns only when the installation is finished. TYPE: `bool \| None` DEFAULT: `True`
`timeout`	Seconds to wait for the installation to finish before raising. Falls back to the engine's default when unset. It also bounds the wait for any environment command already in flight, which happens before the install is submitted and therefore applies even when `await_installation` is `False`. TYPE: `float \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

install_wheel #

install_wheel(
    path: str,
    await_installation: bool | None = True,
    timeout: float | None = None,
)

Install a python library packaged in a wheel file.

import hopsworks

project = hopsworks.login()

# Upload to Hopsworks
ds_api = project.get_dataset_api()
whl_path = ds_api.upload("matplotlib-3.1.3-cp38-cp38-manylinux1_x86_64.whl", "Resources")

# Install
env_api = project.get_environment_api()
env = env_api.get_environment("my_custom_environment")

env.install_wheel(whl_path)

PARAMETER	DESCRIPTION
`path`	The path in Hopsworks where the wheel file is located. TYPE: `str`
`await_installation`	If `True` the method returns only when the installation finishes. TYPE: `bool \| None` DEFAULT: `True`
`timeout`	Seconds to wait for the installation to finish before raising. Falls back to the engine's default when unset. It also bounds the wait for any environment command already in flight, which happens before the install is submitted and therefore applies even when `await_installation` is `False`. TYPE: `float \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.

resolve_npm_package_json #

resolve_npm_package_json(
    path: str | None = None,
    lockfile_path: str | None = None,
    include_dev_dependencies: bool = False,
    *,
    content: str | None = None,
    lockfile_content: str | None = None,
    lockfile_name: str | None = None,
) -> dict

Resolve a package.json's dependencies to the exact versions install_npm takes, without installing.

Pinned versions stand. Where a lockfile is given its recorded versions win, since they are what the project actually ran. Ranges and dist-tags without a lockfile entry resolve to the newest matching registry release. Dependencies that are not registry packages (workspace siblings, npm: aliases, git, file and URL specs) are reported as skipped with the reason.

result = env.resolve_npm_package_json("package.json", lockfile_path="package-lock.json")
for entry in result["packages"]:
    print(entry["name"], entry["requested"], "->", entry.get("version"), entry["status"])

PARAMETER	DESCRIPTION
`path`	The package.json: a path on the local file system, or, when no such local file exists, a path in the project (for example `Resources/my-app/package.json`). TYPE: `str \| None` DEFAULT: `None`
`lockfile_path`	The lockfile next to it, local or in the project: `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock`, `pnpm-lock.yaml` or `bun.lock`. TYPE: `str \| None` DEFAULT: `None`
`include_dev_dependencies`	Whether `devDependencies` are part of the result. TYPE: `bool` DEFAULT: `False`
`content`	The package.json content, instead of `path`. TYPE: `str \| None` DEFAULT: `None`
`lockfile_content`	The lockfile content, instead of `lockfile_path`; needs `lockfile_name`. TYPE: `str \| None` DEFAULT: `None`
`lockfile_name`	The lockfile's file name, which selects its format, when the lockfile is passed as content. TYPE: `str \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`dict`	A dict with `name`, `version`, `packageManager`, `lockfileName`, `warnings` and
`dict`	`packages`, each package carrying `name`, `requested`, `version`, `status`
`dict`	(`RESOLVED`, `SKIPPED` or `UNRESOLVED`), `source`, `dependencyType` and `reason`.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`ValueError`	If neither a path nor content is given.

uninstall #

uninstall(
    library_name: str,
    await_uninstallation: bool | None = True,
    timeout: float | None = None,
    package_source: str | None = None,
) -> None

Uninstall a library from the environment.

import hopsworks

project = hopsworks.login()

env_api = project.get_environment_api()
env = env_api.get_environment("my_custom_environment")

env.uninstall("matplotlib")

PARAMETER	DESCRIPTION
`library_name`	Name of the installed library to remove. TYPE: `str`
`await_uninstallation`	If `True` the method returns only when the uninstallation finishes. TYPE: `bool \| None` DEFAULT: `True`
`timeout`	Seconds to wait for the uninstallation to finish before raising. Falls back to the engine's default when unset. It also bounds the wait for any environment command already in flight, which happens before the removal is submitted and therefore applies even when `await_uninstallation` is `False`. TYPE: `float \| None` DEFAULT: `None`
`package_source`	`PIP` or `NPM`, the ecosystem the library belongs to. Needed when the same name is installed from both PyPI and npm, since the backend refuses to guess which one to remove; a scoped npm name is always npm. TYPE: `str \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.