github-featured / kubeflow--kubeflow

发布

0 发布 121 Git 标签

RSS 订阅源
  • v1.5.0-rc.1 19ae712718
    比较

    Cherry-pick latest commits to the 1.5-branch (#6354)

    frostbyte_neo 发布于 2022-02-11 16:07:40 +00:00

    • Fix(manifests): Upgrade rbac.authorization.k8s.io from v1beta1 to v1 (#6261)

    • proposal: Extend Notebook Controller to expose idleness for Jupyter (#6295)

    • proposal: Extend Notebook Controller to expose idleness for Jupyter (#6270)

    Provide a design doc as a proposal for extending Notebook Controller to
    expose idleness for Jupyter. Our proposal is in markdown format and follows
    the guidelines of the kubeflow/components/proposal/README.md guide.

    You can view the #6270 issue in the following link:
    https://github.com/kubeflow/kubeflow/issues/6270

    Signed-off-by: Athanasios Markou athamark@arrikto.com

    • review: change the title of the proposal

    Change the title of the proposal to only include the
    proposed new feature. The new title of the proposal
    will now be "Expose Idleness Information for Jupyter
    Notebooks".

    • review: rename the proposal markdown file

    We want to give a more specific name to the markdown
    which contains the proposal. Since this proposal
    emphasizes on a feature regarding the Jupyter Notebooks,
    the new name will be:

    20220121-jupyter-notebook-idleness.md

    • Synchronize jupyter-web-application role with clusterrole (#6241)

    • Update role.yaml

    • Update role.yaml

    • Update cluster-role.yaml

    • Kubeflow Roadmap update - with 1.5 details (#6266)

    • Kubeflow Roadmap update - with 1.5 details

    These proposed changes include: identifying that 1.4.1 has been delivered, provides themes for 1.5 and provides details of major features in 1.5 by working group. This is an initial proposal that needs review by the working group leads.

    • correct formatting in KFP features

    Moved KFP features under KFP Control Flow doc

    • updating KFP section

    updating KFP references with updates from KFP team

    • Updated the 1.5 release date to March

    updated the 1.5 release date to March

    • Update ROADMAP.md

    change Hyperparameter leader election to Katib leader election

    Co-authored-by: Andrey Velichkevich andrey.velichkevich@gmail.com

    • Update ROADMAP.md

    improve description and details of feature for metrics collector

    Co-authored-by: Andrey Velichkevich andrey.velichkevich@gmail.com

    • Update Katib description for Early stopping in 1.5

    updating with Andrey's suggestion (but without the world proper). * Validation for Early Stopping algorithm settings helps users to proper reduce model overfitting

    Co-authored-by: Andrey Velichkevich andrey.velichkevich@gmail.com

    • notebooks: Extend Notebook Controller to expose idleness for Jupyter (#6297)

    • notebooks: Update image's tag in make

    Modify Makefile to update properly the TAG
    based on the git TAG.

    Signed-off-by: Athanasios Markou athamark@arrikto.com
    Reviewed-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • notebooks: Expose last-activity

    Extend the notebook-controller to:

    • cull idle Notebook Servers based on their new last-activity
      annotation
    • expose the last activity of each Notebook Server as an annotation
      on the metadata of the corresponding CR object

    Modify notebook_controller.go to:

    • update the Last Activity of each Notebook Server that has a
      Running pod
    • delete the Last Activity Annotation for every Notebook Server
      that does not have a Running pod

    Extend culler.go to:

    • perform culling based on the new last-activity annotation and
      not based on the /api/status endpoint.
    • update the last activity of a Notebook Server, based on the
      kernels' execution states.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Athanasios Markou athamark@arrikto.com

    • notebooks: Introduce a DEV env var

    We introduce a DEV ENV var to allow admins
    develop and test on their local machine their
    custom Notebook Controller.
    We provide information and instructions inside
    the components/notebook-controller/README.md.

    Signed-off-by: Athanasios Markou athamark@arrikto.com
    Reviewed-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • notebooks: Add unit tests for last-activity

    • Introduce new tests for allKernelsAreIdle()

    • Extend the tests for NotebookIsIdle() and for
      NotebookNeedsCulling().

    Signed-off-by: Athanasios Markou athamark@arrikto.com
    Reviewed-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • review: UpdateNotebookLastActivityAnnotation()

    Ensure that UpdateNotebookLastActivityAnnotation() does not return
    "true". This function should not return any value.

    Signed-off-by: Athanasios Markou athamark@arrikto.com

    • jwa: Rework the Storage API of the web app (#6321)

    • wa(back): Add helper for deserializing JSON obj

    In some cases we might need to construct Python k8s lib objects from the
    JSONs that are provided by clients. I.e. the UI will be sending a PVC
    object in json format, so the backend will need to create the
    corresponding client.V1PersistentVolumeClaim object and submit it.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • wa(back): Serialization helper

    Add helper function for converting a k8s-client object into a dict that
    can be sent as an HTTP response.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • wa(back): Add dry run to Notebooks and PVCs

    The backend will need to be able to create objects with dry-run, in
    order to ensure they are valid. The backend will need to check that both
    the Notebook and the PVCs can be created beforehand.

    This way we avoid the scenario where we create PVCs but the Notebook
    fails to be created, and the PVCs are never garbage collected.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • wa(back): Update kubernetes to 0.17

    In order to support dry-run we must use the 0.17 version of the Python
    k8s client.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • wa(back): Extend api module to patch pvcs

    The backend will need to be able to PATCH PVCs in order to set the
    ownerReference to the Notebook that mounts the PVCs.

    Ref: arrikto/dev/issues/386#issuecomment-856700392

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • jwa(back): Work with new Volumes API

    The backend API should not add any more layers of abstractions on top of
    the K8s API. The backend should expect the client/UI to be sending the
    entire PVC spec of a new PVC.

    Refs: arrikto/dev/issues/386

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • jwa(back): Add unittests for new volumes API

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • jwa(back): Extend the PVC info returned

    We want to show both the access mode and size of the existing PVCs, when
    a user clicks on the dropdown to select which PVC to mount.

    The backend will need to provide this information to the frontend. We
    don't want to send the K8s list of PVCs since this will result in a lot
    of unnecessary data to be sent.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Ilias Katsakioris elikatsis@arrikto.com

    • jwa(front): Add proxy config for Rok

    When developing the Rok flavor locally we will need to be able to open
    the Rok chooser. This can be done by using Angular/webpack proxy to
    bring the exposed rok service and the app under the same domain.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Tasos Alexiou tasos@arrikto.com

    • jwa(front): Remove card from form

    The form of the app should not be a big card, but a normal form.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Tasos Alexiou tasos@arrikto.com

    • jwa(front): Install AceModule for yaml editing

    Install AceModule to allow users to edit yamls of objects.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Tasos Alexiou tasos@arrikto.com

    • wa(front): Change the styling of form sections

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Tasos Alexiou tasos@arrikto.com

    • jwa(front): Create common volume components

    Component for:

    • New PVC and configuring its spec
    • Attaching an existing PVC in a Notebook

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Tasos Alexiou tasos@arrikto.com

    • jwa(front): Update Rok form for new Volume API

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Tasos Alexiou tasos@arrikto.com

    • jwa(front): Mark inputs as dirty when restoring Lab

    When the UI autofills the form with values from a JupyterLab snapshot
    then it should mark the touched fields as dirty. This way if a field has
    errors the UI will make that input red.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com
    Reviewed-by: Tasos Alexiou tasos@arrikto.com

    • jwa: Update ConfigMap in manifests

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • jwa(front): Fix format errors

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • profiles: Update the permissions for notebook idleness (#6335)

    Extend the Profiles Controller to give permissions to Notebooks
    controller for making GET requests to notebook's /api/kernels endpoint.

    Refs https://github.com/kubeflow/kubeflow/blob/master/components/proposals/20220121-jupyter-notebook-idleness.md

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • notebooks: Graceful handling of events (#6338)

    • notebooks: Handle events gracefully

    The controller is not exiting the reconciliation loop after it has
    re-emitted a Pod/STS Event as a Notebook Event. This results in the
    controller to later on try and GET a Notebook with the name of the Event
    that triggered the reconciliation loop.

    The controller should exit the reconciliation function once it has
    emitted the event.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • notebooks: Don't reconcile on deleted events

    We don't want to trigger the reconciliation function when an event gets
    deleted.

    If a Notebook would be deleted then the underlying events would
    be deleted as well, which results in the reconcile function to get
    triggered and try to GET Events and Notebooks with the name of the
    deleted event.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • notebooks: Fix endless restarts (#6341)

    • notebooks: Update notebook if timestamp changed

    We don't want to be updating the spec of the notebook if the timestamp
    hasn't changed, since this will lead to constant updates and
    reconciliation loops.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • notebooks: Use a deep-copy of the notebook spec

    The controller should use a deep-copy of the notebook spec when
    calculating the spec for the StatefulSet. If not then we could
    update the notebook object without wanting it, since the spec could have
    been changed when calculating the STS spec.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • notebooks: Add prefix env var only if missing

    The controller should be setting OR updating the NB_PREFIX env var.
    Previously it would always blindly append it to the spec, which could
    result in double entries for the same env var.

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    • releasing: Update tags for v1.5.0-rc.1 (#6343)

    Signed-off-by: Kimonas Sotirchos kimwnasptd@arrikto.com

    Co-authored-by: Andrey Velichkevich andrey.velichkevich@gmail.com

    下载附件