Having re read Chapter one now I've noticed something I didn't really see before. Vernon's main concearn while noticing all the strange happenings is Petunia's reaction. In a way he's more worried how she'll react than his own response. From Pottermore we know that Petunia was hugely jealous of Lily and I get the sense that Vernon knows this and is wanting to keep Petunia from being upset.
I know we're led through the books to hate the Dursleys but with this I almost felt for Vernon and saw in a small way that he might love Petunia and want to protect her.
Seeing all the strange things all Vernon could think about what will the neighbours think would chapter 1 says that the Dursleys were worried what the neighbours would say if the Potters turned up so Vernon is not worried about Petunia he does not like anything different or things doing things they should not being doing .