Preheat oven to 350° F. Butter a 9x5 inch loaf pan and line the bottom with parchment, or leave parchment hanging over the sides if you want easy "handles" to remove the cake from the pan after baking.
In a bowl or liquid measuring cup, mix together wet ingredients (egg yolks, buttermilk, and vanilla).
In a separate bowl, mix together dry ingredients (flours, baking powder, baking soda, and salt).
Cream butter and cream cheese together using a stand mixer fitted with the paddle attachment or electric beaters. Cream until super soft and smooth, 2-3 minutes. Add sugar and lemon zest; beat until combined.
Add about 1/3 of the dry mixture to butter mixture and blend until almost combined. Pour in about 1/2 of the wet mixture; stir until almost combined. Alternate the remaining dry and wet ingredients two more times, just until all ingredients are incorporated together. Do not over-mix!
Fold in huckleberries. Pour cake batter into prepared pan and use the back of a spoon or an offset spatula to smooth it out. Top with lemon streusel, distributing the streusel evenly over the cake batter.
Bake for about 55-65 minutes, or until toothpick inserted in center comes out clean and streusel is lightly browned. Internal temperature of cake should be about 200° F. Allow to cool completely.