github-featured / microsoft--promptflow

发布

0 发布 345 Git 标签

RSS 订阅源
  • promptflow_0.1.0b7.dev1 9b9db376b0
    比较

    Support custom strong type connection (#319)

    frostbyte_neo 发布于 2023-09-21 10:49:41 +00:00

    Description

    This PR does the following things:

    1. Provide generate connection spec and template method when gen tool
      meta (for extension)
    2. Connection CRUD:
      a. create through file
      b. create through PFClient
    3. Submit local flow can succeed with custom strong type connection
    4. Convert connection in flow back to strong type connection when
      executing user tool scripts in tool resolver

    How user writes their own custom strong type connection:

    from promptflow.connections import CustomStrongTypeConnection
from promptflow.contracts.types import Secret

class MyCustomConnection(CustomStrongTypeConnection):
    api_key: Secret
    api_base: str

    Connection spec example:

    {
    'my_tool_package.connections.MyFirstConnection': {
        'connectionCategory': 'CustomKeys',
        'flowValueType': 'CustomConnection',
        'connectionType': 'MyFirstConnection',
        'ConnectionTypeDisplayName': 'MyFirstConnection',
        'configSpecs': [{
                'name': 'api_key',
                'displayName': 'Api Key',
                'configValueType': 'Secret',
                'isOptional': True
            }, {
                'name': 'api_base',
                'displayName': 'Api Base',
                'configValueType': 'str',
                'isOptional': True
            }
        ],
        'module': 'my_tool_package.connections',
        'package': 'my-tools-package-with-cstc',
        'package_version': '0.0.2'
    }
}

    Connection template example:

    {
    "name": "<connection-name>",
    "type": "custom",
    "custom_type": "MyFirstConnection",
    "module": "my_tool_package.connections",
    "package": "my-tools-package-with-cstc",
    "package_version": "0.0.2",
    "configs": {
        "api_base": "<api-base>"
    },
    "secrets": {
        "api_key": "<api-key>"
    },
}

    Create connection through file command:

    pf connection create -f <path-to-my-first-connection>

    CustomConfigs of created connection in DB:

    {
    "api_base": {
        "configValueType": "String",
        "value": "This is my first custom connection."
    },
    "promptflow.custom.connection.custom_type": {
        "configValueType": "String",
        "value": "MyFirstConnection"
    },
    "promptflow.custom.connection.module": {
        "configValueType": "String",
        "value": "my_tool_package.connections"
    },
    "api_key": {
        "configValueType": "Secret",
        "value": "XXX" // real key endswith "=="
    }
}

    Create connection through PFClient:

    client = PFClient()
conn = MyFirstConnection(name=name, secrets={"api_key": "test"}, configs={"api_base": "test"})
client.connections.create_or_update(conn)

    Things to refine:

    1. Functions marked with TODO
    2. Exception handling and error message refine

    Work items yet to do:

    1. Task
      2687123      :
      Promptflow Cli support generate custom strong type connection template
    2. Task
      2691873      :
      Gen custom tool yaml with extended contract
    3. Task
      2691944      :
      Enable schema check for custom strong type connection
    4. Task
      2692760      :
      Support to use custom strong type connection in script tool
    5. Task
      2692758      :
      When gen connection template, also move the comments of connection class
      over

    All Promptflow Contribution checklist:

    • The pull request does not introduce [breaking changes]
    • CHANGELOG is updated for new features, bug fixes or other
      significant changes.
    • I have read the contribution guidelines.

    General Guidelines and Best Practices

    • Title of the pull request is clear and informative.
    • There are a small number of commits, each of which have an
      informative message. This means that previously merged commits do not
      appear in the history of the PR. For more information on cleaning up the
      commits in your PR, see this
      page      .

    Testing Guidelines

    • Pull request includes test coverage for the included changes.

    Co-authored-by: yalu4 yalu4@microsoft.com

    下载附件