Language models are pivotal in natural language processing, forming the foundation for applications in text generation, comprehension, and interaction. However, aligning these models with human ...