There are four major types of mysteries. Within each category, there are sub-categories (like paranormal or historical).
A mystery must have certain elements to be considered a mystery. Essentially, a mystery will have a puzzle or secret, or layers of puzzles or secrets, a setting that fits the type of book, a sound motive, red herrings, and clues.
Most traditionally accepted mysteries have a murder. This is the element that compels people to keep reading. While some mysteries may not have a murder (or murders) at the center, the majority do. If you don’t have a murder, you may be writing a suspense. Something to think about.
Traditional cozies are light, sometimes humorous, slow paced (as compared to the other categories), the murder (usually quite civilized) and sex happen off scene, and the solving of the crime is a battle of wits beteween the reluctant amateur sleuth and the villain. The setting is most often in a small town or community and the subcharacters are quirky and fun. The sleuth falls into the mystery by accident or circumstance and uses common sense/gray cells to solve the crime. Usually first person.
The hardboiled mystery is a detective story with attitude and action. It’s a tough mystery that takes place in a city or urban setting. It’s gritty. It’s violent. The blood and violence (and sex) takes place on screen. Usually the detective is a professional who’s been hired to investigate. Usually first person with a bare-bones or abrupt narrative style. This is not your emotional mystery. Think Raymod Chandler or Michael Connelly.
The soft boiled mystery falls somewhere between the hard boiled and the cozy. It’s not as violent as the hard boiled, but can have more on scene than the cozy. Many soft boiled mysteries have humorous elements. The detective can be a professional or amateur. Misa’s Lola Cruz Mystery Series is an example of soft boiled. Janet Evanovich is also soft boiled (with some caper thrown in).
The detective/sleuth in a police procedural is almost always a law enforcement agent of some sort. The details of the mystery plot are the focus, as opposed to the heavier character development of the other categories. The term police procedural is used because the procdures are detailed and accurate. Rules must be followed and crime details are key. PD James and Tony Hillerman write police procedurals.
Think hard about the kind of details, pov, setting, level of violence in your book and how to categorize it. Not every book fits neatly into a category, but you should be able to see it in one of these categories (even if you have to push or shove a little bit!). Just a caveat, things that aren’t easily marketable–meaning your agent or editor doesn’t know how to explain what it is–are less likely to sell. If you can categorize your book, in general, all the better.