I can’t really answer the question “is there a good reason not to wear it,” but I’d say it’s probably unnecessary. Like other people have said, wearing it can’t hurt your options, but I have a feeling that the admissions interviews will go just as well if you went in a business suit or equivalent. Besides, you probably have the service on your resume, right? So they already know about it, or should - bring a resume along and give it to the guy anyway.
But, on the other side, most of the schools have officers that are there that may want to ask you about what it is you did in the service because they see you in uniform - depends on if you want to deal with that?