Engenharia de prompts para respostas de restaurantes em português: avaliação de tom, empatia, fluência e consistência de marca com LLMs

Abstract

Automating responses to online restaurant reviews requires balancing tone, empathy, fluency, and brand consistency, dimensions still underexplored for large language models (LLMs) applied to Brazilian Portuguese. This work evaluates the viability and maturity of three LLMs (GPT-5.5, Gemini 3.1, and Llama 4) for this task. Drawing on a corpus of 200 real reviews, partitioned into two temporal periods (pre- and post-LLM popularization), a factorial experiment was conducted crossing models, brand personas (formal and informal), and prompt engineering strategies (zero-shot and few-shot). The evaluation followed a mixed-methods design, combining Likert-scale judgments from 15 independent human raters with multidimensional automatic metrics: perplexity (fluency), ToneCal (tone adequacy), stylometric similarity (brand consistency), and the WASSA model (perceived empathy). Results show that no model is universally superior: Llama 4 leads in empathy and tone adequacy, while GPT-5.5 prevails in fluency and brand consistency, with statistically significant differences across models. The zero-shot strategy, supported by well-defined personas, tends to outperform few-shot in pragmatic dimensions. A directional inversion of the persona effect, informal preferred by human raters, yet formal favored by automatic metrics, reveals a systematic bias of computational proxies toward formal register. The main contribution is a multidimensional evaluation framework for text generation in Brazilian Portuguese, with results suggesting that triangulation between human and automatic evaluation is advisable for more robust conclusions in the evaluated scenario.