We want to add a RAG chatbot over our documents. Should we just use a hosted LLM with a vector database?

It depends on the goal. Vector databases are more cost effective and work better when the information is contained and you need deterministic outcomes. LLMs and RAG pipelines work better when there's uncertainty and the results are open, but you can't guarantee the same output for the same input.

For example, for typical sentiment analysis of Amazon comment sections, to understand the general feeling about a product, I'd go with vector search without an LLM.