Kanye West’s controversial comments about slavery were a “turning point” in his failed marriage to Kim Kardashian, the report says