The video, filmed by Smith and obtained by a newspaper, appears to show him laughing while retired gymnast Luke Carson mimics Islamic prayer practices.
Smith, who won pommel horse silver at Rio 2016, later said he was "deeply sorry" for his "thoughtless actions".
British Gymnastics said: "We will be investigating the behaviours reported."
In a statement, the governing body added: "Members who break our code of conduct can face suspension or expulsion from our organisation."
Smith, who has won medals at the past three Olympic Games, has already been censured twice this year by British Gymnastics.
In April, the 27-year-old apologised for questioning the judging at the British Championships, where he was beaten to pommel gold by Max Whitlock.
He was also reprimanded in June for posting an image on social media of an American gymnast, who was 16 when the photograph was taken, accompanied by a comment British Gymnastics said was "unbefitting to a participant".
That reprimand was to remain on his record for two years with the added warning "any further misconduct may lead to even greater consequences".
Smith is one of Britain's best-known gymnasts and won the BBC show Strictly Come Dancing in 2012.
He is taking a break from the sport and is instead touring as a guest celebrity on the Keep Dancing stage production.
Four-time Olympic medallist Louis Smith may face "suspension or expulsion" by British Gymnastics over a video in which he apparently mocks Islam.