Add comprehensive Docker Hub integration with deployment scripts and documentation

DOCKER_HUB_SETUP.md

@@ -0,0 +1,270 @@
+# Docker Hub Setup Guide
+
+This guide will help you deploy your algorithmic trading system to Docker Hub and use it from the cloud.
+
+## Prerequisites
+
+1. **Docker Hub Account**: Create an account at [hub.docker.com](https://hub.docker.com)
+2. **Docker CLI**: Ensure Docker is installed and running
+3. **Repository**: Create a repository on Docker Hub (optional, will be created automatically)
+
+## Quick Start
+
+### 1. Set Environment Variables
+
+```bash
+# Set your Docker Hub username
+export DOCKER_USERNAME=yourusername
+
+# Optionally set your password/token (for automated login)
+export DOCKER_PASSWORD=yourpassword
+```
+
+### 2. Deploy to Docker Hub
+
+```bash
+# Deploy with default settings
+./scripts/docker-build.sh deploy
+
+# Or use the dedicated deployment script
+./scripts/docker-hub-deploy.sh -u yourusername
+
+# Deploy with custom image name and tag
+./scripts/docker-hub-deploy.sh -u yourusername -i my-trading-system -t v1.0.0
+```
+
+### 3. Use from Docker Hub
+
+```bash
+# Start services using Docker Hub images
+./scripts/docker-build.sh hub
+
+# Or manually pull and run
+docker pull yourusername/algorithmic-trading:latest
+docker run -p 8000:8000 yourusername/algorithmic-trading:latest
+```
+
+## Detailed Instructions
+
+### Step 1: Create Docker Hub Account
+
+1. Go to [hub.docker.com](https://hub.docker.com)
+2. Click "Sign Up" and create an account
+3. Verify your email address
+4. Create a repository (optional):
+   - Click "Create Repository"
+   - Name: `algorithmic-trading`
+   - Description: "Algorithmic Trading System with FinRL"
+   - Visibility: Public or Private
+
+### Step 2: Generate Access Token (Recommended)
+
+1. Go to Docker Hub → Account Settings → Security
+2. Click "New Access Token"
+3. Name: `algorithmic-trading-deploy`
+4. Permissions: Read & Write
+5. Copy the token (you won't see it again)
+
+### Step 3: Configure Environment
+
+```bash
+# Add to your ~/.bashrc or ~/.zshrc
+export DOCKER_USERNAME=yourusername
+export DOCKER_PASSWORD=your_access_token
+```
+
+### Step 4: Deploy
+
+```bash
+# Build and deploy
+./scripts/docker-hub-deploy.sh -u yourusername
+
+# The script will:
+# 1. Build the Docker image
+# 2. Run tests (optional)
+# 3. Login to Docker Hub
+# 4. Tag the image
+# 5. Push to Docker Hub
+```
+
+### Step 5: Verify Deployment
+
+1. Check your Docker Hub repository: `https://hub.docker.com/r/yourusername/algorithmic-trading`
+2. You should see your image with the latest tag
+
+## Usage Examples
+
+### Local Development with Docker Hub
+
+```bash
+# Start development environment
+DOCKER_USERNAME=yourusername ./scripts/docker-build.sh hub
+
+# Access services:
+# - Jupyter Lab: http://localhost:8888
+# - Trading System: http://localhost:8000
+```
+
+### Production Deployment
+
+```bash
+# Deploy to production
+docker run -d \
+  --name trading-prod \
+  -p 8000:8000 \
+  -v $(pwd)/data:/app/data \
+  -v $(pwd)/logs:/app/logs \
+  -v $(pwd)/config.yaml:/app/config.yaml:ro \
+  yourusername/algorithmic-trading:latest
+```
+
+### Using Docker Compose
+
+```bash
+# Create .env file
+echo "DOCKER_USERNAME=yourusername" > .env
+echo "TAG=latest" >> .env
+
+# Start services
+docker compose -f docker-compose.hub.yml up -d
+```
+
+## Advanced Usage
+
+### Multiple Tags
+
+```bash
+# Deploy with version tags
+./scripts/docker-hub-deploy.sh -u yourusername -t v1.0.0
+./scripts/docker-hub-deploy.sh -u yourusername -t v1.1.0
+./scripts/docker-hub-deploy.sh -u yourusername -t latest
+```
+
+### Custom Image Names
+
+```bash
+# Deploy with custom name
+./scripts/docker-hub-deploy.sh -u yourusername -i my-trading-bot -t production
+```
+
+### Automated Deployment
+
+```bash
+# Create a deployment script
+cat > deploy.sh << 'EOF'
+#!/bin/bash
+export DOCKER_USERNAME=yourusername
+export DOCKER_PASSWORD=your_token
+./scripts/docker-hub-deploy.sh -u $DOCKER_USERNAME -t $(date +%Y%m%d)
+EOF
+
+chmod +x deploy.sh
+./deploy.sh
+```
+
+## Troubleshooting
+
+### Login Issues
+
+```bash
+# Manual login
+docker login -u yourusername
+
+# Check login status
+docker info | grep Username
+```
+
+### Push Failures
+
+```bash
+# Check if repository exists
+curl https://hub.docker.com/v2/repositories/yourusername/algorithmic-trading/
+
+# Create repository manually if needed
+# Go to Docker Hub → Create Repository
+```
+
+### Permission Issues
+
+```bash
+# Check Docker permissions
+docker ps
+
+# If permission denied, add user to docker group
+sudo usermod -aG docker $USER
+# Log out and back in
+```
+
+### Network Issues
+
+```bash
+# Check Docker Hub connectivity
+curl -I https://hub.docker.com
+
+# Use different DNS if needed
+echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
+```
+
+## Best Practices
+
+### 1. Use Access Tokens
+- Never use your password in scripts
+- Generate access tokens with limited permissions
+- Rotate tokens regularly
+
+### 2. Tag Strategically
+- Use semantic versioning: `v1.0.0`, `v1.1.0`
+- Keep `latest` tag updated
+- Use date tags for testing: `2024-01-15`
+
+### 3. Security
+- Don't commit credentials to git
+- Use environment variables
+- Consider private repositories for sensitive code
+
+### 4. Automation
+- Set up CI/CD pipelines
+- Automate testing before deployment
+- Use GitHub Actions or similar
+
+## CI/CD Integration
+
+### GitHub Actions Example
+
+```yaml
+# .github/workflows/docker-hub.yml
+name: Deploy to Docker Hub
+
+on:
+  push:
+    tags: ['v*']
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Login to Docker Hub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+      - name: Build and push
+        run: |
+          docker build -t ${{ secrets.DOCKER_USERNAME }}/algorithmic-trading:${{ github.ref_name }} .
+          docker push ${{ secrets.DOCKER_USERNAME }}/algorithmic-trading:${{ github.ref_name }}
+```
+
+## Support
+
+- **Docker Hub Documentation**: [docs.docker.com/docker-hub](https://docs.docker.com/docker-hub/)
+- **Repository Issues**: Create an issue in this repository
+- **Community**: Join Docker Hub community forums
+
+## Next Steps
+
+1. **Set up automated deployment** with CI/CD
+2. **Create multiple environments** (dev, staging, prod)
+3. **Monitor usage** with Docker Hub analytics
+4. **Share with team** by adding collaborators
+5. **Scale deployment** to multiple servers 

docker-compose.hub.yml

@@ -0,0 +1,63 @@
+version: '3.8'
+
+services:
+  # Main trading system using Docker Hub image
+  trading-system:
+    image: ${DOCKER_USERNAME:-yourusername}/algorithmic-trading:${TAG:-latest}
+    container_name: algorithmic-trading-hub
+    ports:
+      - "8000:8000"
+    volumes:
+      - ./data:/app/data
+      - ./logs:/app/logs
+      - ./models:/app/models
+      - ./config.yaml:/app/config.yaml:ro
+    environment:
+      - PYTHONPATH=/app
+      - LOG_LEVEL=INFO
+    command: ["python", "-m", "agentic_ai_system.main", "--mode", "live", "--duration", "300"]
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "python", "-c", "import sys; sys.exit(0)"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 40s
+
+  # Development environment with Jupyter Lab
+  development:
+    image: ${DOCKER_USERNAME:-yourusername}/algorithmic-trading:${TAG:-latest}
+    container_name: trading-dev-hub
+    ports:
+      - "8888:8888"
+    volumes:
+      - ./data:/app/data
+      - ./logs:/app/logs
+      - ./models:/app/models
+      - ./config.yaml:/app/config.yaml:ro
+      - .:/app
+    environment:
+      - PYTHONPATH=/app
+      - LOG_LEVEL=DEBUG
+    command: ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root", "--NotebookApp.token=''"]
+    restart: unless-stopped
+
+  # FinRL training service
+  finrl-training:
+    image: ${DOCKER_USERNAME:-yourusername}/algorithmic-trading:${TAG:-latest}
+    container_name: finrl-training-hub
+    volumes:
+      - ./data:/app/data
+      - ./logs:/app/logs
+      - ./models:/app/models
+      - ./config.yaml:/app/config.yaml:ro
+    environment:
+      - PYTHONPATH=/app
+      - LOG_LEVEL=INFO
+    command: ["python", "finrl_demo.py"]
+    restart: "no"
+
+volumes:
+  data:
+  logs:
+  models: 

scripts/docker-build.sh

@@ -108,17 +108,21 @@ show_help() {
     echo "  test        Run tests in Docker"
     echo "  dev         Start development environment"
     echo "  prod        Start production environment"
+    echo "  hub         Start using Docker Hub images"
     echo "  stop        Stop all containers"
     echo "  cleanup     Clean up Docker resources"
     echo "  logs [SVC]  Show logs for a service (default: trading-system)"
     echo "  run CMD     Run a specific command in the container"
+    echo "  deploy      Deploy to Docker Hub (requires docker-hub-deploy.sh)"
     echo "  help        Show this help message"
     echo ""
     echo "Examples:"
     echo "  $0 build"
     echo "  $0 dev"
+    echo "  $0 hub"
     echo "  $0 logs"
     echo "  $0 run 'python demo.py'"
+    echo "  $0 deploy"
 }
 
 # Main script logic
@@ -156,6 +160,27 @@ case "${1:-help}" in
         build_image
         run_command "$2"
         ;;
+    hub)
+        print_status "Starting services using Docker Hub images..."
+        if [ -z "$DOCKER_USERNAME" ]; then
+            print_error "DOCKER_USERNAME environment variable not set"
+            print_status "Please set it: export DOCKER_USERNAME=yourusername"
+            exit 1
+        fi
+        docker compose -f docker-compose.hub.yml up -d
+        print_success "Docker Hub services started"
+        print_status "Trading system available at: http://localhost:8000"
+        print_status "Jupyter Lab available at: http://localhost:8888"
+        ;;
+    deploy)
+        print_status "Deploying to Docker Hub..."
+        if [ -f "scripts/docker-hub-deploy.sh" ]; then
+            ./scripts/docker-hub-deploy.sh "$@"
+        else
+            print_error "docker-hub-deploy.sh not found"
+            exit 1
+        fi
+        ;;
     help|*)
         show_help
         ;;

scripts/docker-hub-deploy.sh

@@ -0,0 +1,215 @@
+#!/bin/bash
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+# Function to print colored output
+print_status() {
+    echo -e "${BLUE}[INFO]${NC} $1"
+}
+
+print_success() {
+    echo -e "${GREEN}[SUCCESS]${NC} $1"
+}
+
+print_warning() {
+    echo -e "${YELLOW}[WARNING]${NC} $1"
+}
+
+print_error() {
+    echo -e "${RED}[ERROR]${NC} $1"
+}
+
+# Default values
+DOCKER_USERNAME=""
+IMAGE_NAME="algorithmic-trading"
+TAG="latest"
+REGISTRY="docker.io"
+
+# Function to show help
+show_help() {
+    echo "Usage: $0 [OPTIONS]"
+    echo ""
+    echo "Options:"
+    echo "  -u, --username USERNAME    Docker Hub username (required)"
+    echo "  -i, --image IMAGE_NAME     Image name (default: algorithmic-trading)"
+    echo "  -t, --tag TAG              Tag (default: latest)"
+    echo "  -r, --registry REGISTRY    Registry URL (default: docker.io)"
+    echo "  -h, --help                 Show this help message"
+    echo ""
+    echo "Examples:"
+    echo "  $0 -u myusername"
+    echo "  $0 -u myusername -i my-trading-system -t v1.0.0"
+    echo ""
+    echo "Environment Variables:"
+    echo "  DOCKER_USERNAME            Set your Docker Hub username"
+    echo "  DOCKER_PASSWORD            Set your Docker Hub password/token"
+}
+
+# Parse command line arguments
+while [[ $# -gt 0 ]]; do
+    case $1 in
+        -u|--username)
+            DOCKER_USERNAME="$2"
+            shift 2
+            ;;
+        -i|--image)
+            IMAGE_NAME="$2"
+            shift 2
+            ;;
+        -t|--tag)
+            TAG="$2"
+            shift 2
+            ;;
+        -r|--registry)
+            REGISTRY="$2"
+            shift 2
+            ;;
+        -h|--help)
+            show_help
+            exit 0
+            ;;
+        *)
+            print_error "Unknown option: $1"
+            show_help
+            exit 1
+            ;;
+    esac
+done
+
+# Check if username is provided
+if [ -z "$DOCKER_USERNAME" ]; then
+    if [ -n "$DOCKER_USERNAME" ]; then
+        DOCKER_USERNAME="$DOCKER_USERNAME"
+    else
+        print_error "Docker Hub username is required!"
+        echo "Use -u option or set DOCKER_USERNAME environment variable"
+        show_help
+        exit 1
+    fi
+fi
+
+# Function to build the image
+build_image() {
+    print_status "Building Docker image..."
+    docker build -t ${IMAGE_NAME}:${TAG} .
+    if [ $? -eq 0 ]; then
+        print_success "Docker image built successfully"
+    else
+        print_error "Failed to build Docker image"
+        exit 1
+    fi
+}
+
+# Function to login to Docker Hub
+login_to_dockerhub() {
+    print_status "Logging in to Docker Hub..."
+    if [ -n "$DOCKER_PASSWORD" ]; then
+        echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
+    else
+        docker login -u "$DOCKER_USERNAME"
+    fi
+    
+    if [ $? -eq 0 ]; then
+        print_success "Successfully logged in to Docker Hub"
+    else
+        print_error "Failed to login to Docker Hub"
+        exit 1
+    fi
+}
+
+# Function to tag the image
+tag_image() {
+    local full_image_name="${REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${TAG}"
+    print_status "Tagging image as: $full_image_name"
+    docker tag ${IMAGE_NAME}:${TAG} "$full_image_name"
+    if [ $? -eq 0 ]; then
+        print_success "Image tagged successfully"
+        echo "$full_image_name"
+    else
+        print_error "Failed to tag image"
+        exit 1
+    fi
+}
+
+# Function to push the image
+push_image() {
+    local full_image_name="${REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${TAG}"
+    print_status "Pushing image to Docker Hub: $full_image_name"
+    docker push "$full_image_name"
+    if [ $? -eq 0 ]; then
+        print_success "Image pushed successfully to Docker Hub!"
+        print_status "You can now pull it with:"
+        echo "  docker pull $full_image_name"
+        print_status "Or use it in docker-compose with:"
+        echo "  image: $full_image_name"
+    else
+        print_error "Failed to push image to Docker Hub"
+        exit 1
+    fi
+}
+
+# Function to run tests before pushing
+run_tests() {
+    print_status "Running tests before deployment..."
+    docker run --rm -v $(pwd):/app ${IMAGE_NAME}:${TAG} pytest -v --tb=short
+    if [ $? -eq 0 ]; then
+        print_success "Tests passed"
+    else
+        print_warning "Some tests failed, but continuing with deployment..."
+    fi
+}
+
+# Function to clean up local images
+cleanup() {
+    print_status "Cleaning up local images..."
+    docker rmi ${IMAGE_NAME}:${TAG} 2>/dev/null || true
+    print_success "Cleanup completed"
+}
+
+# Main deployment process
+main() {
+    print_status "Starting Docker Hub deployment..."
+    print_status "Username: $DOCKER_USERNAME"
+    print_status "Image: $IMAGE_NAME"
+    print_status "Tag: $TAG"
+    print_status "Registry: $REGISTRY"
+    echo ""
+    
+    # Build the image
+    build_image
+    
+    # Run tests (optional)
+    read -p "Run tests before deployment? (y/n): " -n 1 -r
+    echo
+    if [[ $REPLY =~ ^[Yy]$ ]]; then
+        run_tests
+    fi
+    
+    # Login to Docker Hub
+    login_to_dockerhub
+    
+    # Tag the image
+    local full_image_name=$(tag_image)
+    
+    # Push the image
+    push_image
+    
+    # Cleanup
+    read -p "Clean up local images? (y/n): " -n 1 -r
+    echo
+    if [[ $REPLY =~ ^[Yy]$ ]]; then
+        cleanup
+    fi
+    
+    print_success "Deployment completed successfully!"
+    print_status "Your image is now available at:"
+    echo "  https://hub.docker.com/r/${DOCKER_USERNAME}/${IMAGE_NAME}"
+}
+
+# Run main function
+main "$@"