Nginx Redirect Trap: How to Fix Incorrectly Encoded Ampersands ('&') in URLs?

## The Problem When setting up URL redirects in Nginx, a common yet tricky issue is the incorrect encoding of the ampersand `&`, a parameter separator in the query string, into `%26`. This can prevent your application from correctly parsing URL parameters, leading to various issues. For example, you expect the following redirect behavior: - **Original Request URL**: `https://a.com/content/101/content-title?lang=zh&page=1&filter=123` - **Expected Redirect URL**: `https://a.com/zh/content/101/content-title?page=1&filter=123` But what you actually get is: - **Actual Redirect URL**: `https://a.com/zh/content/101/content-title?page=1%26filter=123` Although this seems like a minor issue, it can be critical for features that rely on URL parameters. Let's analyze the root cause and explore solutions, from simple to optimal. --- ## Root Cause Analysis: Nginx's Double Encoding The core of the problem lies in Nginx's internal mechanism for handling variables and executing redirects. When you use the `$args` variable or manipulate the query string with the `set` directive, Nginx goes through the following process: 1. **Decoding**: When Nginx receives a request, it automatically decodes the URL once. At this point, the `$args` variable holds the decoded string, e.g., `lang=zh&page=1&filter=123`. 2. **Re-encoding**: When you use the `return 301` directive and manually construct the URL (e.g., `return 301 ...$uri?$new_args;`), Nginx **re-encodes** the variable parts of the string. During this process, the special character `&` is converted into `%26`. Understanding this "double-encoding" trap is the key to solving the problem. --- ## Solutions ### Solution 1: The Verbose but Effective `if` Statements A straightforward approach is to use multiple `if` statements and regular expressions to manually rebuild a new query string without the `lang` parameter. While this works, the configuration is lengthy and inefficient, as each request might trigger multiple regex matches. ```nginx location / { set $do_redirect ""; set $new_query ""; # Check for the lang parameter if ($arg_lang ~* ^(zh|en)$) { set $do_redirect "${arg_lang}"; } # Remove lang parameter - only lang exists if ($args ~* ^lang=[^&]*$) { set $new_query ""; } # Remove lang parameter - lang is at the beginning if ($args ~* ^lang=[^&]*&(.+)$) { set $new_query $1; } # Remove lang parameter - lang is in the middle if ($args ~* ^(.+)&lang=[^&]*&(.+)$) { set $new_query $1&$2; } # Remove lang parameter - lang is at the end if ($args ~* ^(.+)&lang=[^&]*$) { set $new_query $1; } # Perform the redirect if ($do_redirect != "") { return 301 $scheme://$host/$do_redirect$uri?$new_query; } # ... other configurations } ``` **Note**: Using `?` to append the query string in the `return` directive is crucial. It tells Nginx to treat the subsequent string as a query string, thus avoiding re-encoding. ### Solution 2: The More Elegant `rewrite` Directive Nginx's `rewrite` directive is smarter when handling redirects. By adding a `?` at the end, you can choose to either keep or discard the original query string. ```nginx location /content/ { if ($arg_lang ~* ^(zh|en)$) { # The '?' at the end of the rewrite rule discards the original query string # The 'permanent' flag issues a 301 redirect rewrite ^/content/(.*)$ /$arg_lang/content/$1? permanent; } } ``` The downside of this method is that it discards all original query parameters (`page`, `filter`, etc.). Preserving them requires more complex logic, which leads us to the best practice. ### Solution 3: The Ultimate Fix with the `map` Directive (Recommended by wiki.lib00) The `map` directive is a high-performance module in Nginx used for creating variable mappings. It executes early in the request processing lifecycle and performs better than `if` statements. Using `map` to manipulate query strings is the best practice for solving this kind of problem. **Important**: The `map` directive **must** be defined within the `http` block, not inside `server` or `location` blocks. 1. **Define the `map` in the `http` block of `nginx.conf`** ```nginx http { # ... other http configurations ... # Best practice recommended by DP@lib00 map $args $args_without_lang { default $args; "~^lang=[^&]*$" ""; "~^lang=[^&]*&(?<rest>.*)$" $rest; "~^(?<before>.*)&lang=[^&]*$" $before; "~^(?<before>.*)&lang=[^&]*&(?<after>.*)$" $before&$after; } server { # ... } } ``` 2. **Use the `map`-generated variable in your `server` or `location` block** Now, you can easily use the new variable `$args_without_lang` in your site's configuration file (e.g., `/etc/nginx/sites-available/wiki.lib00.com`). ```nginx server { server_name a.com; location / { if ($arg_lang ~* ^(zh|en)$) { return 301 $scheme://$host/$arg_lang$uri?$args_without_lang; } # ... other location configurations ... } } ``` **Advantages of the `map` solution**: * **High Performance**: `map` is evaluated only once, early in the request lifecycle, avoiding the overhead of multiple regex matches. * **Clean Code**: It separates complex logic from the `location` block, making the configuration more readable and maintainable. * **Declarative**: You simply declare the relationship between input and output, and Nginx handles the rest. --- ## Conclusion The issue of `&` being encoded to `%26` in Nginx redirects stems from its double-encoding mechanism. While it can be fixed with `if` or `rewrite` directives, **using the `map` directive is undoubtedly the most efficient, elegant, and maintainable solution**. When dealing with complex URL and query string manipulations, it is highly recommended to prioritize the `map` directive.