1
155
submitted 11 months ago* (last edited 10 months ago) by ResidualBit@beehaw.org to c/technology@beehaw.org

Regarding Beehaw defederating from lemmy.world and sh.itjust.works, this post goes into detail on the why and the philosophy behind that decision. Additionally, there is an update specific to sh.itjust.works here.

For now, let's talk about what federation is and what defederation means for members of Beehaw or the above two communities interacting with each other, as well as the broader fediverse.

Federation is not something new on the internet. Most users use federated services every day (for instance, the url used to access instances uses a federated service known as DNS, and email is another system that functions through federation.) Just like those services, you elect to use a service provider that allows you to communicate with the rest of the world. That service provider is your window to work with others.

When you federate, you mutually agree to share your content. This means that posting something to a site can be seen by another and all comments are shared. Even users from other sites can post to your site.

Now when you defederate, this results in content to be no longer shared. It didn't reverse any previous sharing or posts, it just stops the information from flowing with the selected instance. This only impacts the site's that are called out.

What this means to you is when a user within one instance (e.g. Beehaw) that's chosen to defederate with another (e.g. lemmy.world), they can no longer interact with content on another instance, and vice versa. Other instances can still see the content of both servers as though nothing has happened.

  • A user is not limited to how many instances they can join (technically at least - some instance have more stringent requirements for joining than others do)
  • A user can interact with Lemmy content without being a user of any Lemmy instance - e.g. Mastodon (UI for doing so is limited, but it is still possible.)

Considering the above, it is important to understand just how much autonomy we, as users have. For example, as the larger instances are flooded with users and their respective admins and mods try to keep up, many, smaller instances not only thrive, but emerge, regularly (and even single user instances - I have one for just myself!) The act of defederation does not serve to lock individual users out of anything as there are multiple avenues to constantly maintain access to, if you want it, the entirety of the unfiltered fediverse.

On that last point, another consideration at the individual level is - what do you want out of Lemmy? Do you want to find and connect with like-minded people, share information, and connect at a social and community level? Do you want to casually browse content and not really interact with anyone? These questions and the questions that they lead to are critical. There is no direct benefit to being on the biggest instance. In fact, as we all deal with this mass influx, figure out what that means for our own instances and interactions with others, I would argue that a smaller instance is actually much better suited for those who just want to casually browse everything.

Lastly, and tangential, another concern I have seen related to this conversation is people feeling afraid of being locked out of the content and conversation from the "main" communities around big topics starting to form across the Lemmiverse (think memes, gaming, tech, politics, news, etc.) Over time, certain communities will certainly become a default for some people just given the community size (there will always be a biggest or most active - it's just a numbers game.) This, again though, all comes down to personal preference and what each individual is looking to get from their Lemmy experience. While there may, eventually, be a “main” sub for (again, by the numbers), there will also always be quite a few other options for targeted discussions on , within different communities, on different instances, each with their own culture and vibe. This can certainly feel overwhelming and daunting (and at the moment, honestly it is.) Reddit and other non-federated platforms provided the illusion of choice, but this is what actual choice looks and feels like.

[edit: grammar and spelling]

2
524
submitted 11 months ago* (last edited 11 months ago) by RedPander@lemmy.rogers-net.com to c/technology@beehaw.org

Hopefully I'm posting this in the right place, but I see Reddit developments as Tech news right now.

Wanted to share a website that is tracking Subreddits that have/will be going dark. It even has a sound notification for when they change their status.

Edit: Adding the stream https://www.twitch.tv/reddark_247

3
1
Online Content Is Disappearing (www.pewresearch.org)
submitted 14 minutes ago by funn@lemy.lol to c/technology@beehaw.org

cross-posted from: https://lemy.lol/post/25166889

4
57
5
13

Canada's industry minister says Ottawa is "considering all measures" after the U.S. announced it would be hiking tariffs on Chinese electric vehicles and other related goods.

François-Philippe Champagne wouldn't rule out Canada imposing similar tariffs during an interview with CBC News Network's Power & Politics on Friday.

"It's fair to say that everything is on the table to protect our industry and our workers," Champagne told host David Cochrane.

"We're working in sync with the United States of America."

President Joe Biden announced earlier this week that the U.S. would be slapping new tariffs on Chinese electric vehicles (EVs), advanced batteries, solar cells, steel, aluminum and medical equipment.

The tariffs are to be phased in over the next three years; those that take effect in 2024 are covering EVs, solar cells, syringes, needles, steel and aluminum and more.

There are currently very few EVs from China in the U.S., but American officials worry that low-priced models made possible by Chinese government subsidies could soon start flooding the U.S. market.

In a separate interview on Tuesday, Flavio Volpe, president of the Automotive Parts Manufacturers' Association, said "Canada has to" implement similar trade levies.

"Now that the Americans have put up a tariff wall, we can't leave the side door open here," Volpe told guest host John Paul Tasker.

Brian Kingston, president of the Canadian Vehicle Manufacturers Association, echoed Volpe's argument in a post on X, formerly Twitter.

"Canada cannot be out of step with the U.S. on China. We need aligned policies that strengthen the North American auto supply chain," he wrote.

Champagne insisted that Canada wouldn't be a route for China to gain access to the North American EV market.

"Canada has never been and will never be a backdoor [for] China in the North American market and our U.S. friends understand that," he said.

The federal government has partnered with provinces to attract investments from major automotive manufacturers to spur electric vehicle production in Canada.

The same day the U.S. announced its new tariffs, Asahi Kasei Corp., in partnership with Honda, announced the construction of a $1.6-billion electric vehicle battery plant in Port Colborne, Ont.

Volpe said domestic EV production could be held back if China floods the Canadian market with cheaper products.

"There's no logic for Canada to force our market to electrify and then turn the market over to the Chinese," he said.

China has maintained that the U.S. tariffs are a violation of international trade rules. It is not clear how the country will respond at this point.

Volpe suggested Beijing could retaliate by implementing export controls on its critical minerals that are used in EV battery manufacturing.

Champagne said it's important for Canada to shore up its own critical mineral production.

On Thursday, Canada and the U.S. announced they would be co-investing in critical mineral producers for the first time as they work to boost regional supplies.

Natural Resources Canada and the U.S. Department of Defense are together putting about $32.5 million into Fortune Minerals Ltd. — which is working on a project with bismuth and cobalt in the Northwest Territories — and Lomiko Metals Inc., focused on a graphite project in Quebec.

6
36

lmao imagine that

7
137
submitted 19 hours ago by floofloof@lemmy.ca to c/technology@beehaw.org
8
65

By Tinglong Dai, Bernard T. Ferrari Professor of Business, Johns Hopkins University

In June 2019, then-presidential candidate Joe Biden tweeted: “Trump doesn’t get the basics. He thinks his tariffs are being paid by China. Any freshman econ student could tell you that the American people are paying his tariffs.”

Fast-forward five years to May 2024, and President Biden has announced a hike in tariffs on a variety of Chinese imports, including a 100% tariff that would significantly increase the price of Chinese-made electric vehicles.

For a nation committed to reducing greenhouse gas emissions, efforts by the U.S. to block low-cost EVs might seem counterproductive. At a price of around US$12,000, Chinese automaker BYD’s Seagull electric car could quickly expand EV sales if it landed at that price in the U.S., where the cheapest new electric cars cost nearly three times more.

As an expert in global supply chains, however, I believe the Biden tariffs can succeed in giving the U.S. EV industry room to grow. Without the tariffs, U.S. auto sales risk being undercut by Chinese companies, which have much lower production costs due to their manufacturing methods, looser environmental and safety standards, cheaper labor and more generous government EV subsidies.

Tariffs have a troubled history

The U.S. has a long history of tariffs that have failed to achieve their economic goals.

The Smoot-Hawley Tariff Act of 1930 was meant to protect American jobs by raising tariffs on imported goods. But it backfired by prompting other countries to raise their tariffs, which led to a drop in international trade and deepened the Great Depression.

Biden speaks at a podium with people standing behind him holding United Steelworkers signs.

President George W. Bush’s 2002 steel tariffs also led to higher steel prices, which hurt industries that use steel and cost American manufacturing an estimated 200,000 jobs. The tariffs were lifted after the World Trade Organization ruled against them.

The Obama administration’s tariffs on Chinese-made solar panels in 2012 blocked direct imports but failed to foster a domestic solar panel industry. Today, the U.S. relies heavily on imports from companies operating in Southeast Asia – primarily Cambodia, Malaysia, Thailand and Vietnam. Many of those companies are linked to China.

Why EV tariffs are different this time

Biden’s EV tariffs, however, might defy historical precedent and succeed where the solar tariff failed, for a few key reasons:

1. Timing matters.

When Obama imposed tariffs on solar panels in 2012, nearly half of U.S. installations were already using Chinese-manufactured panels. In contrast, Chinese-made EVs, including models sold in the U.S. by Volvo and Polestar, have negligible U.S. market shares.

Because the U.S. market is not dependent on Chinese-made EVs, the tariffs can be implemented without significant disruption or price increases, giving the domestic industry time to grow and compete more effectively.

By imposing tariffs early, the Biden administration hopes to prevent the U.S. market from becoming saturated with low-price Chinese EVs, which could undercut domestic manufacturers and stifle innovation.

2. Global supply chains are not the same today.

The COVID-19 pandemic exposed vulnerabilities in global supply chains, such as the risk of disruptions in the availability of critical components and delays in production and shipping. These issues prompted many countries, including the U.S., to reevaluate their dependence on foreign manufacturers for critical goods and to shift toward reshoring – bringing manufacturing back to the U.S. – and strengthening domestic supply chains.

The war in Ukraine has further intensified the separation between U.S.-led and China-led economic orders, a phenomenon I call the “Supply Chain Iron Curtain.”

In a recent McKinsey survey, 67% of executives cited geopolitical risk as the greatest threat to global growth. In this context, EVs and their components, particularly batteries, are key products identified in Biden’s supply chain reviews as critical to the nation’s supply chain resilience.

Ensuring a stable and secure supply of these components through domestic manufacturing can mitigate the risks associated with global supply chain disruptions and geopolitical tensions.

3. National security concerns are higher.

Unlike solar panels, EVs have direct national security implications. The Biden administration considers Chinese-made EVs a potential cybersecurity threat due to the possibility of embedded software that could be used for surveillance or cyberattacks.

U.S. Commerce Secretary Gina Raimondo has discussed espionage risks involving the potential for foreign-made EVs to collect sensitive data and transmit it outside the U.S. Officials have raised concerns about the resilience of an EV supply chain dependent on other countries in the event of a geopolitical conflict.

BYD targets EV sales in Mexico

While Biden’s EV tariffs might succeed in keeping Chinese competition out for a while, Chinese EV manufacturers could try to circumvent the tariffs by moving production to countries such as Mexico.

This scenario is similar to past tactics used by Chinese solar panel manufacturers, which relocated production to other Asian countries to avoid U.S. tariffs.

Chinese automaker BYD, the world leader in EV sales, is already exploring establishing a factory in Mexico to produce its new electric truck. Nearly 10% of cars sold in Mexico in 2023 were produced by Chinese automakers.

Given the changing geopolitical reality, Biden’s 100% EV tariffs are likely the beginning of a broader strategy rather than an isolated measure. U.S. Trade Representative Katherine Tai hinted at this during a recent press conference, stating that addressing vehicles made in Mexico would require “a separate pathway” and to “stay tuned” for future actions.

Is Europe next?

For now, given the near absence of Chinese-made EVs in the U.S. auto market, Biden’s EV tariffs are unlikely to have a noticeable short-term impact in the U.S. They could, however, affect decisions in Europe.

The European Union saw Chinese EV imports more than double over a seven-month period in 2023, undercutting European vehicles by offering lower prices. Manufacturers are concerned. When finance ministers from the Group of Seven advanced democracies meet in late May, tariffs will be on the agenda.

Biden’s move might encourage similar protective actions elsewhere, reinforcing the global shift toward securing supply chains and promoting domestic manufacturing.

9
37
submitted 23 hours ago by 0x815@feddit.de to c/technology@beehaw.org

Targeting posts boasting of personal wealth appears to be part of campaign to ‘purify the internet cultural environment’.

Chinese social media companies have launched a new crackdown on user content, targeting posts that show off personal wealth and financial extravagance.

In a statement posted online on Wednesday, Weibo said it had spent this month carrying out special management work on “undesirable value-orientated content”, including content “showing off wealth and worshipping money”.

The statement said it had targeted posts showing off luxury cars and expensive properties. Posts seen as bragging about wealth and the freedom that comes with being rich were also removed.

Other social media companies, including Tencent, Douyin and Xiaohongshu, posted similar statements.

The crackdown is a part of China’s campaign to create a “social-ecological environment that is civilised, healthy and harmonious”, Weibo’s statement said. It encouraged users to instead create or share high-quality, truthful and positive value-oriented content on the platform, to further create “a good community atmosphere of upward mobility and goodness”.

Douyin said it had removed 4,701 messages and 11 accounts from 1 to 7 May. Xiaohongshu said it had cleaned up 4,273 “illegal” posts in the past two weeks and closed 383 accounts, and Weibo said it had removed more than 1,100 pieces of content, according to Chinese media outlet, The Cover.

The stricter approach appears to be part of the Chinese authorities’ nationwide campaign to “purify the internet cultural environment”, which began in 2016.

Despite the Chinese Communist party’s efforts to achieve a “common prosperity”, the gap between the rich and the poor is increasing.

Data released from China’s National Bureau of Statistics showed the 2023 income gap in Beijing has reached its greatest value since data collection began in 1985. The share of China’s national income earned by the top 10% of the population has increased from 27% in 1978 to 41% in 2015, nearing the US’s 45% and surpassing France’s 32%, according to the Stanford Center on China’s Economy and Institutions.

Policies and crackdowns on social behaviour that the ruling Chinese Communist party has deemed unacceptable have also been seen offline.

In September 2023 Beijing amended laws to prohibit the comments, clothing and symbols that “hurt national sentiments”. In 2022, sports administrators said they would ban new tattoos for national football team members, and advised those with them already to have them removed or cover them up.

In August 2020 the Chinese government launched “Operation empty plate”, a campaign to stop food and beverage waste and cultivate frugality. And in 2018 the government called for “comprehensive reform” of the wedding industry to end to “vulgar wedding practices” like expensive wedding gifts, lavish ceremonies and demands for increasingly high bride prices.

In 2022 the Chinese National Radio and Television Administration attracted controversy when it said it was determined to crack down on the plastic surgery and “sissy” aesthetic on TV.

10
73
11
34
submitted 1 day ago by 0x815@feddit.de to c/technology@beehaw.org

Taiwan's National Security Bureau (NSB) official Ko Cheng-heng said that Beijing’s campaign against Taiwan would spark a strong backlash from democracies around the world.

Cyberattacks from China have surged to 2.5 million per day as it intensifies its “gray zone” activities ahead of president-elect William Lai’s (賴清德) inauguration on Monday next week, the National Security Bureau (NSB) said yesterday at a meeting of the legislatures’ Foreign Affairs and National Defense Committee.

An official in charge of the agency’s fifth division made the comments in response to questions from Democratic Progressive Party (DPP) Legislator Wang Ting-yu (王定宇) about possible Chinese activities that would coincide with the inauguration.

China has increased disinformation operations and cyberattacks, the official said, adding that the latter more than doubled the average of 1 million hacks per day earlier this year.

The attacks, which mainly targeted government agencies, are being countered by the bureau’s efforts to find and eliminate cybersecurity vulnerabilities before they could be exploited, they said.

The bureau detected an uptick in Chinese “gray zone” warfare, but nothing unusual from the Chinese People’s Liberation Army (PLA) in the months leading to the inauguration, NSB Deputy Director-General Ko Cheng-heng (柯承亨) said.

Asked by DPP Legislator Michelle Lin (林楚茵) whether Wednesday’s sanctions against Taiwanese pundits by China threatened to “Hong Kongize” Taiwan, Ko said that Beijing’s campaign against the nation would spark a stronger backlash from the world’s democracies.

Vice Minister of Foreign Affairs Remus Chen (陳立國) said that China’s experiment with extraterritorial repression has angered democratic nations globally, which cannot tolerate infringement on their national sovereignty and their citizens’ rights.

Meanwhile, a US Department of State spokesperson on Wednesday urged China to stop military, diplomatic and economic pressure against Taiwan, calling for Beijing to conduct meaningful dialogue across the Taiwan Strait.

The spokesperson made the remark in response to a request for comment from Central News Agency (CNA).

On Tuesday and Wednesday, the Ministry of National Defense detected 51 PLA vessels operating near Taiwan.

The US expresses its concern over China’s continued attempt at intimidating and pressuring Taiwan, as Beijing’s actions carry a risk of miscalculation harming regional peace, the US spokesperson said.

Washington is to continue its opposition to any unilateral change of the “status quo” and support the peaceful resolution of the question concerning both sides of the Strait, positions that agree with the interests of Taiwanese, they said.

The US would ensure that diplomatic and military communication channels with Beijing remain open during the sensitive period of Taiwan’s transfer of power, a US official separately told CNA.

Likewise, Washington will maintain unofficial ties with Taiwan, they said.

The American Institute in Taiwan has advised Lai’s incoming administration of Washington’s long-standing policy on affairs in the Strait, they added.

The US cautions Taiwan that China would likely carry out coercive actions via “gray zone” tactics, they said, citing the example of China Coast Guard pressure near Kinmen County in February.

These tactics are not new for Beijing, which has practiced them in the South China Sea and around the Diaoyutai Islands (釣魚台), they said.

12
174
13
43
submitted 1 day ago* (last edited 1 day ago) by hedge@beehaw.org to c/technology@beehaw.org
14
26
submitted 1 day ago* (last edited 1 day ago) by 0x815@feddit.de to c/technology@beehaw.org

According to the European consumer protection group BEUC, the Chinese online retailer Temu "fails to provide sufficient traceability of the traders that sell on its platform" and thereby fail "to ensure that the products sold to EU consumers conform to EU law", BEUC said in a release.

"Temu is using manipulative practices such as dark patterns to get consumers, for example, to spend more than they might originally want to, or to complicate the process of closing down their account", BEUC adds, and it fails to "provide transparency about how it recommends products to consumers".

As a result, BEUC filed a complaint with the European Commission, while several of BEUC’s national members filed the same complaint with their competent national authorities, namely Austria, Belgium, Bulgaria, Czech Republic, Denmark, Finland, France, Greece, Italy, Netherlands, Poland, Portugal, Slovakia, Spain, Sweden, and Luxembourg.

[Edit typo.]

15
56
16
37
submitted 2 days ago by funn@lemy.lol to c/technology@beehaw.org

cross-posted from: https://lemy.lol/post/25062075

17
47
submitted 2 days ago* (last edited 2 days ago) by hedge@beehaw.org to c/technology@beehaw.org

I was under the impression that Privacy Badger wasn't considered useful any more . . . ? They should've just recommended using Firefox instead, yes?

EDIT: They spoke to, but IMHO, did not give enough time to, Cory Doctorow and Brewster Kahle. They mentioned Mastodon 👍, and described the Fediverse while not actually calling it that! A bit frustrating.

18
34
submitted 2 days ago by hedge@beehaw.org to c/technology@beehaw.org

I've never completely understood this, but I think the answer would probably be "no," although I'm not sure. Usually when I leave the house I turn off wifi and just use mobile data (this is a habit from my pre-VPN days), although I guess I should probably just keep it on since using strange Wi-Fi with a VPN is ok (unless someone at Starbucks is using the evil twin router trick . . . ?). I was generally under the impression that mobile data is harder to interfere with than Wi-Fi, but I could well be wrong and my notions out of date. So, if need be, please set me straight. 🙂

19
113
submitted 2 days ago by alyaza@beehaw.org to c/technology@beehaw.org
20
87

Archive.org link

Some highlights I found interesting:

After Tinucci had cut between 15% and 20% of staffers two weeks earlier, part of much wider layoffs, they believed Musk would affirm plans for a massive charging-network expansion.

Musk, the employees said, was not pleased with Tinucci’s presentation and wanted more layoffs. When she balked, saying deeper cuts would undermine charging-business fundamentals, he responded by firing her and her entire 500-member team.

The departures have upended a network widely viewed as a signature Tesla achievement and a key driver of its EV sales.

Despite the mass firings, Musk has since posted on social media promising to continue expanding the network. But three former charging-team employees told Reuters they have been fielding calls from vendors, contractors and electric utilities, some of which had spent millions of dollars on equipment and infrastructure to help build out Tesla’s network.

Tesla's energy team, which sells solar and battery-storage products for homes and businesses, was tasked with taking over Superchargers and calling some partners to close out ongoing charger-construction projects, said three of the former Tesla employees.

Tinucci was one of few high-ranking female Tesla executives. She recently started reporting directly to Musk, following the departure of battery-and-energy chief Drew Baglino, according to four former Supercharger-team staffers. They said Baglino had historically overseen the charging department without much involvement from Musk.

Two former Supercharger staffers called the $500 million expansion budget a significant reduction from what the team had planned for 2024 - but nonetheless a challenge requiring hundreds of employees.

Three of the former employees called the firings a major setback to U.S. charging expansion because of the relationships Tesla employees had built with suppliers and electric utilities.

21
135
submitted 3 days ago by Hirom@beehaw.org to c/technology@beehaw.org
22
160

I hate to go as cliche as "surprising absolutely no one," but really, this is not a surprise.

23
32

This is the alternative Invidious link for the embedded article.

By Mayank Kejriwal, Research Assistant Professor of Industrial & Systems Engineering, University of Southern California.

Many people understand the concept of bias at some intuitive level. In society, and in artificial intelligence systems, racial and gender biases are well documented.

If society could somehow remove bias, would all problems go away? The late Nobel laureate Daniel Kahneman, who was a key figure in the field of behavioral economics, argued in his last book that bias is just one side of the coin. Errors in judgments can be attributed to two sources: bias and noise.

Bias and noise both play important roles in fields such as law, medicine and financial forecasting, where human judgments are central. In our work as computer and information scientists, my colleagues and I have found that noise also plays a role in AI.

Noise in this context means variation in how people make judgments of the same problem or situation. The problem of noise is more pervasive than initially meets the eye. A seminal work, dating back all the way to the Great Depression, has found that different judges gave different sentences for similar cases.

Worryingly, sentencing in court cases can depend on things such as the temperature and whether the local football team won. Such factors, at least in part, contribute to the perception that the justice system is not just biased but also arbitrary at times.

Other examples: Insurance adjusters might give different estimates for similar claims, reflecting noise in their judgments. Noise is likely present in all manner of contests, ranging from wine tastings to local beauty pageants to college admissions.

Noise in the data

On the surface, it doesn’t seem likely that noise could affect the performance of AI systems. After all, machines aren’t affected by weather or football teams, so why would they make judgments that vary with circumstance? On the other hand, researchers know that bias affects AI, because it is reflected in the data that the AI is trained on.

For the new spate of AI models like ChatGPT, the gold standard is human performance on general intelligence problems such as common sense. ChatGPT and its peers are measured against human-labeled commonsense datasets.

Put simply, researchers and developers can ask the machine a commonsense question and compare it with human answers: “If I place a heavy rock on a paper table, will it collapse? Yes or No.” If there is high agreement between the two – in the best case, perfect agreement – the machine is approaching human-level common sense, according to the test.

So where would noise come in? The commonsense question above seems simple, and most humans would likely agree on its answer, but there are many questions where there is more disagreement or uncertainty: “Is the following sentence plausible or implausible? My dog plays volleyball.” In other words, there is potential for noise. It is not surprising that interesting commonsense questions would have some noise.

But the issue is that most AI tests don’t account for this noise in experiments. Intuitively, questions generating human answers that tend to agree with one another should be weighted higher than if the answers diverge – in other words, where there is noise. Researchers still don’t know whether or how to weigh AI’s answers in that situation, but a first step is acknowledging that the problem exists. Tracking down noise in the machine

Theory aside, the question still remains whether all of the above is hypothetical or if in real tests of common sense there is noise. The best way to prove or disprove the presence of noise is to take an existing test, remove the answers and get multiple people to independently label them, meaning provide answers. By measuring disagreement among humans, researchers can know just how much noise is in the test.

The details behind measuring this disagreement are complex, involving significant statistics and math. Besides, who is to say how common sense should be defined? How do you know the human judges are motivated enough to think through the question? These issues lie at the intersection of good experimental design and statistics. Robustness is key: One result, test or set of human labelers is unlikely to convince anyone. As a pragmatic matter, human labor is expensive. Perhaps for this reason, there haven’t been any studies of possible noise in AI tests.

To address this gap, my colleagues and I designed such a study and published our findings in Nature Scientific Reports, showing that even in the domain of common sense, noise is inevitable. Because the setting in which judgments are elicited can matter, we did two kinds of studies. One type of study involved paid workers from Amazon Mechanical Turk, while the other study involved a smaller-scale labeling exercise in two labs at the University of Southern California and the Rensselaer Polytechnic Institute.

You can think of the former as a more realistic online setting, mirroring how many AI tests are actually labeled before being released for training and evaluation. The latter is more of an extreme, guaranteeing high quality but at much smaller scales. The question we set out to answer was how inevitable is noise, and is it just a matter of quality control?

The results were sobering. In both settings, even on commonsense questions that might have been expected to elicit high – even universal – agreement, we found a nontrivial degree of noise. The noise was high enough that we inferred that between 4% and 10% of a system’s performance could be attributed to noise.

To emphasize what this means, suppose I built an AI system that achieved 85% on a test, and you built an AI system that achieved 91%. Your system would seem to be a lot better than mine. But if there is noise in the human labels that were used to score the answers, then we’re not sure anymore that the 6% improvement means much. For all we know, there may be no real improvement.

On AI leaderboards, where large language models like the one that powers ChatGPT are compared, performance differences between rival systems are far narrower, typically less than 1%. As we show in the paper, ordinary statistics do not really come to the rescue for disentangling the effects of noise from those of true performance improvements. Noise audits

What is the way forward? Returning to Kahneman’s book, he proposed the concept of a “noise audit” for quantifying and ultimately mitigating noise as much as possible. At the very least, AI researchers need to estimate what influence noise might be having.

Auditing AI systems for bias is somewhat commonplace, so we believe that the concept of a noise audit should naturally follow. We hope that this study, as well as others like it, leads to their adoption.

24
27
submitted 3 days ago by 0x815@feddit.de to c/technology@beehaw.org

The authors introduce and evaluate an open-source software package and methodological framework for detecting and analysing coordinated behaviour on social media, namely the Coordination Network Toolkit, utilising weighted, directed multigraphs to capture intricate coordination dynamics.

To whom it may concern.

25
73
submitted 4 days ago* (last edited 4 days ago) by Powderhorn@beehaw.org to c/technology@beehaw.org
view more: next ›

Technology

37202 readers
1104 users here now

Rumors, happenings, and innovations in the technology sphere. If it's technological news or discussion of technology, it probably belongs here.

Subcommunities on Beehaw:


This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.

founded 2 years ago
MODERATORS