Saying Kaddish for Asher

I didn't want to go to synagogue this morning and say kaddish for Asher on the fourth anniversary of his death, but I did go.  We lit a candle for him last night, and today we plan to visit his grave.
I didn't want to go, because I have found it very difficult to pray in the past few years, to feel that I am praying to some entity that hears my prayers, cares whether or not I prayer, or has any interest at all in human beings.
I didn't want to go  because I almost never attend morning services during the week, and it feels hypocritical to me to attend them only when I have a death to commemorate.
Would Asher want me to say kaddish for him?  I guess so.  Although I do not believe in the afterlife in any serious way, I can't deny that at certain moments I feel Asher's presence around me.
There is some consolation in prayer with a group of other men, which is what orthodox Jewish prayer is, some of whom are also there because they are in mourning.  Bereavement is isolating, and isolation compounds grief.
If I hadn't gone, I would have felt guilty.  But, having gone, I don't feel satisfied in any way, just less guilty.
The four years that have gone by have only made Asher more dead, and the pain I feel is less acute, because I am learning to live with it.  But sometimes a searing memory suddenly whips me, and I miss him terribly.  Often it is at otherwise happy moments, when I wish Asher could share them.