Putting It All Together: Implementing a Complete Preflight Checks System

We've now explored ten different preflight checks for securing AI applications. In this final article, let's bring everything together and discuss how to implement a complete, well-organized system.

Recap: The Checks We've Covered

Our preflight checks fall into several categories:

1. Input Validation

   • Language detection
   • Input length validation

2. Rate Limiting

   • IP-based rate limiting
   • User-based rate limiting
   • Global rate limiting

3. Content Safety

   • Blacklisted keywords
   • OpenAI moderation

4. AI Detection

   • AI-generated content detection

5. Architecture

   • The need for preflight checks
   • Balancing freedom and safety
   • Tiered architecture

The Complete Implementation

Let's see how all these checks come together in a well-organized system:

// preflight-checks.ts
import { PreflightContext, PreflightResult, PreflightCheck } from './types';

// Import checks from each category
import { inputLengthCheck, languageCheck } from './checks/input-validation';
import {
  enhancedRateLimitCheck,
  rateLimitUserCheck,
  rateLimitGlobalCheck,
} from './checks/rate-limiting';
import { blacklistedKeywordsCheck, contentModerationCheck } from './checks/content-safety';
import { aiDetectionCheck } from './checks/ai-detection';

// Define check tiers based on performance characteristics
const tier1Checks: PreflightCheck[] = [
  // Fast, simple checks first
  inputLengthCheck,
  languageCheck,
  blacklistedKeywordsCheck,
];

const tier2Checks: PreflightCheck[] = [
  // Local rate limiting
  enhancedRateLimitCheck,
  rateLimitUserCheck,
  rateLimitGlobalCheck,
];

const tier3Checks: PreflightCheck[] = [
  // External API calls
  contentModerationCheck,
];

const tier4Checks: PreflightCheck[] = [
  // Computationally intensive checks
  aiDetectionCheck,
];

// Combine all checks in order
const allChecks: PreflightCheck[] = [
  ...tier1Checks,
  ...tier2Checks,
  ...tier3Checks,
  ...tier4Checks,
];

// Main function to run all preflight checks
export async function runPreflightChecks(context: PreflightContext): Promise<PreflightResult[]> {
  const results: PreflightResult[] = [];
  let allPassed = true;

  console.log(`Running preflight checks for: ${context.userId || 'anonymous'}`);

  // Process all checks in sequence, stopping at first failure
  for (const check of allChecks) {
    try {
      console.log(`Running check: ${check.name}`);

      // Run the check
      const result = await check.run(context);

      // Add the result to our collection
      results.push(result);

      // If the check failed, stop processing
      if (!result.passed) {
        console.warn(`Preflight check failed: ${check.name} - ${result.code}`);
        allPassed = false;
        break;
      }
    } catch (error) {
      console.error(`Error in preflight check ${check.name}:`, error);

      // Add error result
      results.push({
        passed: false,
        code: `${check.name}_unexpected_error`,
        message: 'Unexpected error in preflight check',
        details: { error: error instanceof Error ? error.message : 'Unknown error' },
        severity: 'error',
      });

      allPassed = false;
      break;
    }
  }

  // Log the final outcome
  if (allPassed) {
    console.log('All preflight checks passed');
  } else {
    console.warn('Preflight checks failed');
  }

  return results;
}

// Helper function to track warnings
export async function processPreflightResults(
  results: PreflightResult[],
  context: PreflightContext
): Promise<void> {
  // Find any results that should increment warning count
  const warningResults = results.filter((result) =>
    ['blacklisted_keywords', 'moderation_flagged', 'ethical_concerns'].includes(result.code)
  );

  // If there are warning results and we have an IP, increment warning count
  if (warningResults.length > 0 && context.ip && context.ip !== 'unknown') {
    try {
      // Import here to avoid circular dependencies
      const { incrementWarningCount } = await import('./checks/enhanced-rate-limit-check');
      await incrementWarningCount(context.ip);
    } catch (error) {
      console.error('Failed to increment warning count:', error);
    }
  }
}

This implementation:

1. Organizes checks into performance-based tiers
2. Runs checks in sequence, stopping at the first failure
3. Provides detailed logging and error handling
4. Tracks warnings for potentially problematic content

Using the Preflight Checks System

Here's how to use this system in your API routes:

// example-api-route.ts
import {
  runPreflightChecks,
  processPreflightResults,
} from '../lib/preflight-checks/preflight-checks';
import { PreflightContext } from '../lib/preflight-checks/types';

export async function POST(req: Request) {
  try {
    // Extract data from request
    const data = await req.json();
    const { message } = data;

    // Get user information
    const userId = getUserIdFromRequest(req);
    const ip = getIpFromRequest(req);
    const userAgent = req.headers.get('user-agent') || 'unknown';

    // Prepare preflight context
    const context: PreflightContext = {
      lastMessage: message,
      userId,
      ip,
      userAgent,
    };

    // Run preflight checks
    const preflightResults = await runPreflightChecks(context);

    // Process any warning results
    await processPreflightResults(preflightResults, context);

    // Check if all preflight checks passed
    const allPassed = preflightResults.every((result) => result.passed);

    if (!allPassed) {
      // Get the first failed check result
      const failedResult = preflightResults.find((result) => !result.passed);

      // Return error response
      return new Response(
        JSON.stringify({
          error: true,
          message: failedResult?.message || 'Preflight checks failed',
          code: failedResult?.code || 'unknown_error',
          details: failedResult?.details || {},
        }),
        { status: 400 }
      );
    }

    // All checks passed, proceed with AI request
    const aiResponse = await callAIService(message, userId);

    // Return successful response
    return new Response(
      JSON.stringify({
        message: aiResponse,
      }),
      { status: 200 }
    );
  } catch (error) {
    console.error('API route error:', error);

    // Return server error
    return new Response(
      JSON.stringify({
        error: true,
        message: 'Server error processing request',
        details: { error: error instanceof Error ? error.message : 'Unknown error' },
      }),
      { status: 500 }
    );
  }
}

Performance Optimization Strategies

To optimize performance, consider these strategies:

1. Check Ordering

Order your checks by:

1. Speed: Fastest checks first
2. Reliability: Most reliable checks first
3. Cost: Free checks before paid API calls

2. Caching Results

For some checks, cache results to avoid redundant processing:

import { redis } from '../utils/redis';

// Example of caching moderation results
async function getCachedModerationResult(content: string) {
  // Create a hash of the content
  const contentHash = createHash(content);

  // Try to get from cache
  const cached = await redis.get(`moderation:${contentHash}`);

  if (cached) {
    return JSON.parse(cached);
  }

  return null;
}

async function cacheModerationResult(content: string, result: any) {
  // Create a hash of the content
  const contentHash = createHash(content);

  // Cache for 24 hours
  await redis.set(`moderation:${contentHash}`, JSON.stringify(result), 'EX', 86400);
}

3. Parallel Execution for Independent Checks

For checks that don't depend on each other, consider parallel execution:

// Run independent checks in parallel
async function runIndependentChecks(context: PreflightContext) {
  // These checks can run in parallel
  const [lengthResult, languageResult, blacklistResult] = await Promise.all([
    inputLengthCheck.run(context),
    languageCheck.run(context),
    blacklistedKeywordsCheck.run(context),
  ]);

  // Process results
  const results = [lengthResult, languageResult, blacklistResult];
  const allPassed = results.every((result) => result.passed);

  return { results, allPassed };
}

4. Smart Rate Limiting

Implement dynamic rate limiting based on system load:

// Adjust rate limits based on system load
async function getDynamicRateLimit() {
  // Get current system metrics
  const metrics = await getSystemMetrics();

  // Calculate appropriate rate limit
  if (metrics.cpuLoad > 80) {
    return 50; // Reduced limit under high load
  } else {
    return 100; // Normal limit
  }
}

Best Practices

Based on our exploration, here are key best practices:

1. Defense in Depth

Don't rely on a single check. Use multiple checks that address different attack vectors:

Input Validation → Rate Limiting → Content Safety → AI Detection

2. Fail Closed for Critical Checks

For critical security checks, fail closed (reject on error):

catch (error) {
  console.error('Critical check error:', error);

  // Fail closed for security checks
  return {
    passed: false,
    code: 'security_error',
    message: 'Unable to verify content safety',
    severity: 'error'
  };
}

3. Detailed Logging

Implement comprehensive logging to detect patterns and issues:

// Log all check results
function logPreflightResults(results: PreflightResult[], context: PreflightContext) {
  const userId = context.userId || 'anonymous';
  const ip = context.ip || 'unknown';

  console.log({
    timestamp: new Date().toISOString(),
    userId,
    ip,
    results: results.map((r) => ({
      check: r.code,
      passed: r.passed,
      severity: r.severity,
    })),
  });
}

4. User Feedback

Provide clear, helpful feedback when checks fail:

function getUserFriendlyMessage(code: string): string {
  const messages = {
    input_too_short: 'Please provide a longer message to get a meaningful response.',
    input_too_long: 'Your message is too long. Please break it into smaller chunks.',
    unsupported_language:
      'This service currently only supports English. Please submit your request in English.',
    ip_rate_limited: "You've made too many requests recently. Please try again later.",
    user_rate_limited: "You've reached your usage limit. Please try again later.",
    global_rate_limited: 'Our service is experiencing high demand. Please try again later.',
    blacklisted_keywords: 'Your message contains prohibited content. Please revise and try again.',
    moderation_flagged:
      'Your message was flagged by our content filter. Please revise and try again.',
    ai_generated_content: 'We detected AI-generated content. Please submit your own message.',
  };

  return messages[code] || "Your request couldn't be processed. Please try again.";
}

5. Regular Updates

Keep your checks updated as new evasion techniques emerge:

// Version your check configurations for easier updates
const checkVersions = {
  blacklistedKeywords: '2025.03',
  aiDetection: '2025.02',
  contentModeration: '2025.03',
};

// Function to check if updates are needed
async function checkForUpdates() {
  const latestVersions = await fetchLatestCheckVersions();

  for (const [check, version] of Object.entries(checkVersions)) {
    if (latestVersions[check] !== version) {
      console.log(`Update available for ${check}: ${version} → ${latestVersions[check]}`);
    }
  }
}

Monitoring and Analytics

Implement a monitoring system to track the performance and effectiveness of your checks:

// Example metrics to track
interface PreflightMetrics {
  totalRequests: number;
  checkFailures: Record<string, number>;
  averageProcessingTime: number;
  checkProcessingTimes: Record<string, number>;
  costIncurred: number;
}

// Update metrics after each request
async function updateMetrics(results: PreflightResult[], startTime: number) {
  const processingTime = Date.now() - startTime;

  await redis.incr('metrics:total_requests');
  await redis.incrby('metrics:total_processing_time', processingTime);

  // Track failures by check
  for (const result of results) {
    if (!result.passed) {
      await redis.incr(`metrics:failures:${result.code}`);
    }
  }
}

Conclusion

Throughout this series, we've built a comprehensive system of preflight checks that balances security, performance, and user experience. From simple input validation to sophisticated AI content detection, these checks work together to create a robust defense system for your AI application.

Remember that security is not a one-time implementation but an ongoing process. As AI models and attack techniques evolve, so too should your preflight checks. Regular updates, monitoring, and refinement are essential to maintaining an effective security posture.

By implementing the preflight checks we've explored, you're taking a significant step toward building a responsible AI application that protects both your users and your resources.

Thanks for following along with this series, and best of luck in securing your AI applications!