GCP publish python package in production

GCP: Publish Python Package in Production

This guide explains how to use Google Artifact Registry to manage shared Python code as a package. This approach eliminates code duplication between your Cloud Functions and server.

Step 1: Structure Your Shared Code

Create a new Python package for your shared logic (e.g., common_logic).

common_logic/
├── setup.py
├── common_logic/
│   ├── __init__.py

Step 2: Create setup.py

Define your package configuration in a setup.py file:

common_logic/
├── setup.py
├── common_logic/
│   ├── __init__.py

Step 3: Set Up Google Artifact Registry

1. Enable the Artifact Registry API:

from setuptools import setup, find_packages

setup(
    name="common_logic",
    version="0.1.0",
    packages=find_packages(),
    install_requires=[
        "pandas>=1.3.0",
    ],
    author="Your Name",
    author_email="your.email@example.com",
    description="Common logic for app",
)

1. Create a Python repository:

   gcloud services enable artifactregistry.googleapis.com

Step 4: Configure Authentication

1. Create a service account:

   gcloud artifacts repositories create python-packages \
       --repository-format=python \
       --location=us-central1 \
       --description="Python packages repository"

1. Grant necessary permissions:

   gcloud iam service-accounts create artifact-publisher \
       --description="Service account for publishing to Artifact Registry"

1. Create and download a key:

   gcloud artifacts repositories add-iam-policy-binding python-packages \
       --location=us-central1 \
       --member="serviceAccount:artifact-publisher@${PROJECT_ID}.iam.gserviceaccount.com" \
       --role="roles/artifactregistry.writer"

Step 5: Build and Upload Package

1. Install build tools:

   gcloud iam service-accounts keys create key.json \
       --iam-account=artifact-publisher@${PROJECT_ID}.iam.gserviceaccount.com

1. Build the package:

   pip install build twine

1. Configure twine for Artifact Registry:

   python -m build

1. Upload the package:

   cat > ~/.pypirc << EOL
   [distutils]
   index-servers = common-logic-repo
   [common-logic-repo]
   repository: https://us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/
   username: _json_key_base64
   password: $(base64 -w0 key.json)
   EOL

Step 6: Use the Package

In Cloud Functions

1. Create a requirements.txt file:

   twine upload --repository common-logic-repo dist/*

1. Use the package in your Cloud Function:

   --index-url https://pypi.org/simple
   --extra-index-url https://oauth2accesstoken:${ARTIFACT_REGISTRY_TOKEN}@us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/simple/
   common-logic==0.1.0

In Server Code

1. Add to your server's requirements.txt:

   from common_logic import ...

   def cloud_function(request):
       # Your cloud function code using the imported functions
       pass

1. Use it in your server code:

   --index-url https://pypi.org/simple
   --extra-index-url https://oauth2accesstoken:${ARTIFACT_REGISTRY_TOKEN}@us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/simple/
   common-logic==0.1.0

Step 7: CI/CD Integration

1. Add the service account key as a secret in your GitHub repository.
2. Update your Cloud Build configuration:

   from common_logic import ...
   # Your server code using the imported functions

Step 8: Version Management

1. Update the version in setup.py.
2. Build and upload the new version.
3. Update requirements.txt in both Cloud Functions and server code.
4. Deploy both components.

Best Practices

• Use semantic versioning for your package.
• Pin specific versions in requirements.txt.
• Test new versions thoroughly before deploying.
• Keep a changelog of version changes.
• Use environment variables for PROJECT_ID and LOCATION.
• Include comprehensive documentation in your package.

Common Issues and Solutions

Authentication Errors

• Verify service account permissions.
• Ensure key.json is properly encoded.
• Check .pypirc configuration.

Package Not Found

• Verify repository URL format.
• Check if the package was successfully uploaded.
• Ensure requirements.txt uses the correct URL format.

Version Conflicts

• Pin specific versions of dependencies.
• Use virtual environments for testing.
• Document dependency requirements clearly.

The above is the detailed content of GCP publish python package in production. For more information, please follow other related articles on the PHP Chinese website!