Mattis is scheduled to meet Afghanistan President Ashraf Ghani to exchange views on matters relating to mutual interests, including the war on terror and the peace process in Afghanistan.
However, the insurgents, who seized a district centre in western Afghanistan earlier this week, have given no public sign of accepting Ghani's offer, instead issuing several statements suggesting they meant to keep fighting.
Ghani's peace plan includes eventually recognising the Taliban as a political party.
However, the Taliban appears likely to miss that conference and have so far ruled out direct talks with the Western-backed government in Kabul, which they say is an illegitimate, foreign-imposed regime.
The Afghan government and the Taliban held peace talks in 2015, but they broke down nearly immediately.
The group ruled Afghanistan until 2001 when it was defeated by a United States -led troops in the wake of the September 11 attacks and it is fighting to restore Islamic rule in the country.
"We want the Afghans to lead and provide the substance to the reconciliation effort", he said.
"What does that victory look like?"
"It's all working to achieve a political reconciliation, not a military victory", Mattis said.
As part of the so-called South Asia Strategy, President Donald Trump a year ago ordered the increased bombing of Taliban targets - including drug-making labs and training camps.
It marks Mattis' third visit to the country, where about 11,000 US troops are stationed.
Approximately 14,000 American forces are now in Afghanistan, up from a low of about 8,500 when Obama left office.
Mattis said the jump in attacks on civilians was an indication that a pressured Taliban is unable to conduct broader, ground-taking operations.
"We've had some groups of Taliban - small groups - who have either started to come over or expressed an interest in talking". While the U.S. military says the strategy has hit the Taliban hard, they still control or contest much of the country.
Previous peace offers to the militants were refused.
Mattis also said he had seen some changes in Pakistan's behaviour since Trump blasted the country previous year for harbouring the Taliban.