Are your group of friends considered active? Or are you the only active person?
I have some friends that are active and some friends that are like me, we're OK with chilling at a cafe and etc. However we do make times to hike, outing, trips, and other activities. So it's a good healthy balance for everyone. If your friends are non-active, then it might be a bit difficult to convince them. But you should try nonetheless.
'Hey, we should xx. This place has a shop we can stop by afterwards' Etc. So this would include an activity PLUS a cafe nearby for them to stop by. This is how I usually pitched it, so everyone gets an activity they like. If all else fail, there's nothing wrong with doing things alone.
The other way to look at it is, they have probably done alot of activities by themselves or with other people already, and so when they meet you they are tired and only wanted to stop at cafes and talk.