Microsoft has confirmed that they're offering refunds to Xbox users who are only subscribed to Gold so that they can access the apps. As part of the recent announcement that there'll be a Kinect-less Xbox One, the platform holder also announced that Games with Gold is coming to Xbox One, and confirmed that access to the Xbox's apps would no longer require a Gold subscription.
Since then an updated FAQ has confirmed that those that are paying for Gold just so that they can access apps will be able to get a refund if they want to cancel their subscription when the new update rolls out next month. Here's the exact wording on the FAQ:
"Once the Xbox One and Xbox 360 system updates become available in June, Xbox Live Gold members who purchased a paid membership before that day can cancel and receive a pro-rata refund of any unused remaining days between the date of cancellation and the date their paid Gold membership ends. Cancellation and pro-rata refund requests must be made by August 31, 2014 and require six to eight weeks for processing. Free or trial Gold memberships are not eligible for a refund."