By Se Young Lee and Stephen Nellis SEJONG, South Korea/SAN FRANCISCO (Reuters) – South Korea's antitrust regulator fined Qualcomm Inc 1.03 trillion won ($854 million) for what it called unfair business practices in patent licensing and modem chip sales, a decision the U.S. chipmaker said it will challenge in court. The fine, the largest ever levied in South Korea, marks the latest antitrust setback for Qualcomm's most profitable business of licensing wireless patents to the mobile industry, at a time when the business is facing headwinds from a cooling smartphone market. The Korea Fair Trade Commission (KFTC) ruled on Wednesday Qualcomm abused its dominant market position and forced handset makers to pay royalties for an unnecessarily broad set of patents as part of sales of its modem chips.