No, you simply add VAT onto your invoice you issue to your customers, and then pay it over to the government at the end of each VAT period. Before paying it over, you can deduct any VAT you have paid on any business expenses you have incurred, so you can actually make money from being VAT registered.
As an alternative to the traditional method of VAT accounting (above), the flat rate scheme allows you to pay VAT as a flat rate percentage on your gross sales inclusive of VAT. If you register under this scheme you will not reclaim VAT on your normal business expenses. This is only likely to be advantageous where you have a low turnover and minimal business expenses.