The accusations prompted Cboe Global Markets, the financial exchange operator that is home to the Cboe Volatility Index <.VIX>, to ask Wall Street's self-funded regulator, the Financial Industry Regulatory Authority (FINRA), to look into the matter, two sources familiar with the situation said.

"Cboe has a dedicated regulatory department that works with FINRA to monitor certain trading activity for our securities markets, including trading activity that could impact the VIX settlement,” Greg Hoogasian, Cboe’s chief regulatory officer, told Reuters in a statement.

The letter to the U.S. Securities and Exchange Commission and the Commodity Futures Trading Commission from Zuckerman Law, a Washington-based firm, said it had been written on behalf of a person, not identified, who had held senior roles in the investment sector.

In addition to the letter, the law firm said it had filed a formal complaint with regulators on behalf of the unidentified client. It was not clear from the letter and a redacted copy of the complaint seen by Reuters what evidence the person might have provided to support claims of manipulation.

The complaint says regulators should look at trading data for proof of alleged manipulation.

The letter, dated Monday, alleged that trading firms had taken advantage of the way the VIX is calculated in order to manipulate the index, costing investors nearly $2 billion a year.

Financial products that seek to track the VIX are at the center of recent stock market turbulence.

The VIX estimates the expected near-term volatility conveyed by S&P 500 <.SPX> index option prices. The CBOE calculates an official settlement monthly that determines whether large blocks of VIX futures expire worthless or turn a profit.

The whistleblower's letter alleges a flaw in the calculation of the VIX that allows trading firms to manipulate the VIX index by posting quotes for S&P 500 options, without actually trading.

Cboe spokeswoman Suzanne Cosgrove said in a statement, "This letter is replete with inaccurate statements, misconceptions and factual errors, including a fundamental misunderstanding of the relationship between the VIX Index, VIX futures and volatility exchange traded products, among other things."

The SEC and CFTC declined to comment.

William Speth, vice president and head of research at Cboe, said, "There are structural safeguards built into the process of the calculation of the VIX settlement value that would hinder the type of manipulation the letter alleges."

"Our regulatory group actively surveils for potential VIX settlement manipulation," he said.

CLAIMS NOT NEW

Last year John Griffin and Amin Shams of the McCombs School of Business at the University of Texas, Austin wrote a paper noting significant spikes in trading volume in S&P 500 index options at the exact time of the monthly VIX settlement.

The complaint alleging manipulation referenced the paper. (http://bit.ly/2HbqSYR)

"It's exactly what we talked about in our paper," Griffin told Reuters. "I would think the Cboe would want to redesign the settlement process to make it fairer and more transparent."

Cboe disagreed with the paper's conclusions when it was posted on a website for research and said the jump in activity around settlement was due to investors adjusting their positions around contract expirations.

Cboe's Speth said that traders posting quotes on SPX options have to keep them open for 10 minutes and could be forced to complete the trade.

"And in a world were microseconds count, that's an eternity," he said.

Stuart John Barton, portfolio manager at Invest In Vol, a volatility-focused registered investment advisor in Stamford, Connecticut, said, "There has been a longstanding understanding that there is a potential for the manipulation of the index level."

Barton said that losses to investors from such manipulations are probably much smaller than the nearly $2 billion annually alleged in the letter.

The letter alleged that manipulation of the volatility index also played a role in last week's market turmoil where VIX-related exchange traded products (ETP) experienced big price swings.

Both Barton and Griffon cautioned that it was unlikely the type of manipulation alleged in the letter could have caused such a large spike in VIX futures.

"It would be scaremongering to say that it had a big impact on the reason why these products blew up," Barton said.

(Reporting Saqib Iqbal Ahmed and John McCrank; Additional reporting by Tevor Hunnicutt in New York, Rama Venkat Raman in Bengaluru and Pete Schroeder in Washington; Editing by Daniel Bases and Lisa Shumaker)

By Saqib Iqbal Ahmed and John McCrank

Stocks treated in this article : CBOE Holdings, SPDR S&P 500 ETF Trust