Introduction

In today’s fast-paced development environment, teams often grapple with time-consuming and inconsistent code reviews. This can lead to bottlenecks, delayed releases, and a dip in code quality. Imagine a solution that automates this crucial process, providing instant, consistent, and intelligent feedback on every pull request. This guide will walk you through building just such a system: an AI-powered code review bot leveraging GitHub Actions, Slack, and Large Language Models (LLMs). Get ready to transform your code review workflow and enhance team efficiency!

What You’ll Discover

By following this comprehensive tutorial, you will learn how to:

• Automatically trigger AI-driven code reviews for every new pull request.
• Deliver insightful review results directly to Slack, complete with interactive buttons for quick actions.
• Seamlessly integrate GitHub, Slack, and your backend to create a robust, automated pipeline.
• Harness the power of LLMs (such as OpenAI or Gemini) to provide advanced code intelligence and analysis.

How the AI Code Review Bot Works

Our AI-powered code review bot operates on a streamlined architecture designed for maximum efficiency:

1. A developer opens or updates a pull request on GitHub.
2. This action triggers a GitHub Action workflow.
3. The GitHub Action extracts the code diff and sends it to our backend service.
4. The backend service utilizes an LLM to analyze the code diff and generate review comments.
5. Review summaries and interactive prompts are then posted to a designated Slack channel.
6. Team members can approve or request changes directly from Slack, which updates the GitHub PR via the backend.

(Image: System Architecture Overview)

Core Technologies Utilized

To bring this powerful bot to life, we will be using a combination of modern technologies:

• GitHub Actions: For orchestrating automated workflows.
• Next.js: As an example codebase for demonstration.
• Node.js/Express Backend: To handle Slack interactivity and integrate with LLMs.
• Slack App: For real-time notifications and interactive user engagement.
• Cloudflare Tunnel (or ngrok): To securely expose your local development server to the internet for Slack callbacks.
• OpenAI (or other LLM provider like Gemini): The brain behind the intelligent code review analysis.

Prerequisites for Building Your Bot

Before diving into the implementation, ensure you have the following ready:

• A GitHub account for repository management.
• A Slack workspace for team communication and bot interaction.
• Node.js (v18 or higher) installed on your development machine.
• A GitHub personal access token with repo and pull_requests scopes.
• A Slack bot token with chat:write, commands, and interactivity permissions.
• An OpenAI API key (or a key from your preferred LLM provider).
• (Optional but recommended): ngrok or Cloudflare Tunnel for convenient local testing.

Project Folder Structure

Understanding the project layout is key to navigating the codebase. Here’s a glance at the bot’s architecture:

code-review-llm/
├── .github/
│   └── workflows/
│       └── code-review.yml         # GitHub Actions workflow for AI code review
├── src/
│   └── services/
│       ├── reviewRunner.js         # Main script to run LLM code review
│       ├── llmService.js           # LLM API integration
│       ├── reviewService.js        # Review logic
│       └── ...                     # Other utilities/services
├── public/
│   └── vite.svg                    # Static assets
├── App.jsx, main.jsx, App.css      # Frontend entry and styles
├── package.json, README.md         # Project metadata and docs
└── slack-backend/
    ├── index.js                    # Express backend for Slack interactivity
    ├── package.json                # Backend dependencies
    ├── .env                        # Backend environment variables (not committed)
    └── README.md                   # Backend setup instructions

Step-by-Step Implementation Guide

1. Set Up the GitHub Action

This workflow triggers on pull requests, initiates the AI review, and posts the results to Slack.

• Navigate to your GitHub repository settings and add OPENAI_API_KEY and SLACK_BOT_TOKEN as repository secrets.

(Image: GitHub Action Setting)

• Create the code-review.yml file within .github/workflows/ with the following content:

name: AI Code Review
on:
  pull_request:
    types: [opened, synchronize, reopened]
jobs:
  code-review:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
      actions: read
      issues: read
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
          token: ${{ secrets.GITHUB_TOKEN }}
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'npm'
      - name: Install dependencies
        run: npm ci
      - name: Get PR diff
        id: diff
        run: |
          git fetch origin ${{ github.event.pull_request.base.ref }}
          DIFF=$(git diff origin/${{ github.event.pull_request.base.ref }}...HEAD --unified=3)
          echo "diff<<EOF" >> $GITHUB_OUTPUT
          echo "$DIFF" >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT
      - name: Run AI Code Review
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          REPO_OWNER: ${{ github.repository_owner }}
          REPO_NAME: ${{ github.event.repository.name }}
          PR_DIFF: ${{ steps.diff.outputs.diff }}
        run: node src/services/reviewRunner.js
      - name: Send review summary log to Slack
        env:
          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
        run: |
          SUMMARY=$(cat review-summary.txt 2>/dev/null || echo "No summary file found.")
          curl -X POST https://slack.com/api/chat.postMessage \
            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
            -H "Content-type: application/json" \
            --data "{\"channel\": \"test-bot\", \"text\": \"$SUMMARY\"}"
      - name: Send LLM review to Slack with buttons
        env:
          SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
        run: |
          curl -X POST https://slack.com/api/chat.postMessage \
            -H "Authorization: Bearer $SLACK_BOT_TOKEN" \
            -H "Content-type: application/json" \
            --data '{
              "channel": "test-bot",
              "text": "Code review for PR #${{ github.event.pull_request.number }}: ${{ github.event.pull_request.title }}",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*Code Review Request*\n*PR #${{ github.event.pull_request.number }}:* ${{ github.event.pull_request.title }}\n\n*LLM Review:*\nReview output here\n\nPlease review and take action below:"
                  }
                },
                {
                  "type": "actions",
                  "elements": [
                    {
                      "type": "button",
                      "text": { "type": "plain_text", "text": "✅ Approve" },
                      "value": "approve_${{ github.event.pull_request.number }}",
                      "action_id": "approve_action",
                      "style": "primary"
                    },
                    {
                      "type": "button",
                      "text": { "type": "plain_text", "text": "💬 Request Changes" },
                      "value": "comment_${{ github.event.pull_request.number }}",
                      "action_id": "comment_action",
                      "style": "danger"
                    }
                  ]
                }
              ]
            }'

2. Develop the Backend for Slack Interactivity

This Node.js/Express backend will process button clicks from Slack and interact with the GitHub API to update PRs. Ensure you use a dedicated GitHub bot account’s API token for these operations.

app.post('/slack/interactive', async (req, res) => {
  const payload = JSON.parse(req.body.payload);
  const { actions, user, response_url } = payload;
  const action = actions[0];
  const prNumber = action.value.replace(/^(approve|comment)_/, '');
  if (action.action_id === 'approve_action') {
    // Logic to approve PR via GitHub API goes here
  }
  // Add logic for 'comment_action' (requesting changes)
});

To enable Slack to communicate with your local backend during development, you’ll need to expose your local server.

• Install Cloudflare Tunnel globally: npm install -g cloudflared
• Run the tunnel: cloudflared tunnel --url http://localhost:3000
• Configure your Slack App’s “Interactivity & Shortcuts” section with the generated tunnel URL as the Request URL.

(Image: Slack Request URL)

3. Test Your Bot in Action

To see your AI code review bot come to life:

• Create a new branch in your GitHub repository.
• Make some code changes and push a commit to this branch.
• Open a pull request.
• Observe the GitHub Action running.
• Soon after, you will see a review summary posted to Slack by your bot.

(Image: AI code review summary)

• The bot will also post a detailed LLM review with interactive buttons in Slack, allowing you to either “Approve” or “Request Changes.”

(Image: Slack bot message)

• Clicking on these buttons will trigger the backend, updating your GitHub PR accordingly.

(Image: Approve or Request Changes)

Next Steps & Enhancements

This bot provides a strong foundation. Consider these ideas for further development:

• Expand Slack actions to include custom comments, assigning reviewers, or triggering specific CI/CD pipelines.
• Integrate the bot with other popular chat platforms like Discord or Microsoft Teams.
• Enhance LLM prompts for more nuanced and context-aware feedback.

GitHub Repository

Access the complete source code for this project on GitHub:

• https://github.com/mrdaiking/code-review-llm

Conclusion

You have successfully built an intelligent, AI-powered code review bot that promises to significantly cut down review times, ensure consistency, and boost overall team productivity. This is just the beginning of what’s possible with automation and AI in development. We encourage you to customize this bot to fit your team’s specific needs and share your innovative results!

(Original publisher credit: This post was originally published on TechByCuong. If you found this guide helpful, please share it with your network and leave a comment below! Happy coding! 🚀)