github-featured / kubeflow--kubeflow

发布

0 发布 121 Git 标签

RSS 订阅源
  • v1.6.0-rc.0 411342f2bd
    比较

    Introduce a mechanism to build all Kubeflow images (#6555)

    frostbyte_neo 发布于 2022-07-01 20:11:00 +00:00

    • build: Update components makefiles for building

    We'll create a top-level Makefile under components/ dir
    that has the following rules:

    • build-all:
      To build all images locally

    • push-all:

      • We can use a specific REGISTRY and retag the images
      • Push all the images

    This top-level Makefile will run the sub-Makefiles that every
    component has for building and pushing the images.

    We modified every sub-Makefile as follows:

    • We don't use a registry in images by default

    • Removed unused rules and vars

    • Use the --dirty flag of git describe in TAG

        --dirty[=<mark>]
         Describe the working tree. It means describe HEAD and
         appends <mark> (-dirty by default) if the working tree
         is dirty.

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • build: Create makefiles for notebook servers

    The common starting point of building the notebook-server
    images are the following 4 base images:

    • base
    • codeserver
    • jupyter
    • rstudio

    All other server images need to build on top of them. We'll
    dynamically pass the base images in every Dockerfile by
    using an ARG IMG. We can set the value of this ARG during
    docker build with the --build-arg CLI argument.

    This way we build both the base images with a tag locally,
    and then we pass that image as arg via the Makefile and build the rest

    So we modified our building procedure as follows:

    1. Build the base image since everything starts from there

    2. Pass the base image as an ARG in the Dockerfiles of
      jupyter, codeserver, rstudio images and build on top

    3. Pass the base images in all other server images and build
      on top

    For that we will:

    1. Create a Makefile for each of the notebook servers, in each folder
      a. Each makefile will be responsible for building the bases and use args for passing them on

    2. Use the central Makefile to call each Makefile from above

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • cherry-pick: Notebook server upstream fixes

    Relevant upstream PR: https://github.com/kubeflow/kubeflow/pull/6466/files

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • build: Fix Makefiles
    • Remove build-gcp and build-gcr rules as we don't use them anywhere in
      the project
    • Fix code conficts

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • build: Fix Dockerfiles for notbook-server images

    We remove the previous logic of using already built images as bases.
    The users must use only the Makefiles to pass the appropriate BASE_IMG
    and build the images correctly.

    Thus, we have Makefiles everywhere that:

    • Can build any base image
    • If an image requires another notebook base, then we first build that one using its makefile,
      and then use it as docker ARG for building the next one

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • build: Fix notebook-controller Makefile

    Removed a misplaced "|" char that breaks the Makefile

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • build: Update GH action workflows

    • Update workflow for notebook-server images:

      • Add a step for building all images by using the
        central-Makefile under components/example-notebook-servers/ dir.
      • Add a step for pushing all images by using the
        central-Makefile under components/example-notebook-servers/ dir.

    • Update workflow for all Kubeflow images:

      • Add a step for building & pushing all images by using the
        top-level Makefile under components/ dir.

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • build: Remove completely ECR references from images

    Replace everywhere the "public.ecr.aws/j1r0q0g6/notebooks/notebook-servers"
    prefix with "kubeflownotebookswg"

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    • build: Fix GH actions for Kubeflow components

    Fix GH actions to use the updated make rules
    when building the Kubeflow component images.

    Remove the "docker.io" prefix when building with
    GH action workflows

    Signed-off-by: Apotolos Gerakaris apoger@arrikto.com

    下载附件