.Net: Fix #6030 - Mitigating Prompt Injection in Liquid Templates

[LittleLittleCloud]

Motivation and Context

#6030

Description

In this implementation, the Ġ will be reserved in liquid template which is used to replace : in all input variables when unsafe content is not allowed.

The encoding process for input variables when unsafe content is not allowed is
- replace all : to Ġ // this is the extra step comparing with HandlerBar Template
- Encode xml using HttpUtility.HtmlEncode

The decoding process is
- replace all Ġ to :

This PR introduces a new process to mitigate potential prompt injection attacks from input variables when using liquid templates. Here's a breakdown of the steps:

Before rendering, each input variable undergoes a transformation: all occurrences of :are replaced with :. This ensures that message tags like system:, user:, or assistant: are not present if AllowUnsafeContent is set to false. No replacement occurs if AllowUnsafeContent is true.
After rendering, each message content is processed based on the AllowUnsafeContent setting. If it's false, all : instances are reverted back to :, followed by calling html_encode on each message content. If AllowUnsafeContent is true, only html_encode is called. This additional encoding step is necessary because ChatPromptParser always decodes XML message content, requiring the liquid template to undergo an extra encoding step to ensure the rendered content matches the original before rendering.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[stephentoub]

In this implementation, the Ġ will be reserved in liquid template which is used to replace : in all input variables when unsafe content is not allowed.

Why is that necessary? And why is that ok? Why are we in a position to effectively ban "Ġ"?