Custom display template behavior on page refresh - asp.net-mvc-3

Question

I am having an issue with my custom template when page is refreshed. My scenario is that I have multiple address records to be displayed, in which I opted to use custom templates so I won't have to use foreach() to display each record. Here's my custom template code:
#model Address
<div>
#Html.LabelFor(model => model.AddressLine)
#Html.DisplayFor(model => model.AddressLine)
<br />
</div>
Let's say I have 2 address records for display. What's happening is that upon initial page load, it will display 2 records correctly but when page is refreshed, it will display the 2 records again in which now I end up with 4 records on screen. Subsequent page refreshes will display even more duplicates on the screen. My question is, do I need to handle that page refresh event? Or am I just doing something very obviously wrong here?
UPDATE
As requested here is my code for the View and the Action,
View:
#model Patient
<div id="editDetails">
#using(Html.BeginForm("Edit", "Patient"))
{
#Html.DisplayFor(m => m.Addresses)
<input type="submit" value="Submit" />
}
</div>
Action:
public ActionResult Details(string id)
{
Patient patient = patientRepository.GetPatientData(id);
if (patient != null)
{
patientData.GetOtherDetails(patient);
return View(patient);
}
else
return View();
}